[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:09 dist-test-slave-kvfs krb5kdc[20914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:09 dist-test-slave-kvfs krb5kdc[20914](info): set up 2 sockets
Aug 08 00:42:09 dist-test-slave-kvfs krb5kdc[20914](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 00:42:11.205252 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.459s user 0.002s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:11 dist-test-slave-kvfs krb5kdc[20914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:11.367980 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:32937
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:32937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:11.473908 20934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:11.474239 20934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:11.474287 20934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:11.475696 20934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:11.475876 20934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:11.475947 20934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:11.475992 20934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:11.476043 20934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:11.478169 20934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:32937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:32937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:11.478701 20934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:11.479171 20934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:42:11.487143 20934 server_base.cc:1034] running on GCE node
W20260808 00:42:11.487143 20941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:11.488152 20942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:11.488478 20945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:11.488907 20934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:11.489399 20934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:11.490597 20934 hybrid_clock.cc:648] HybridClock initialized: now 1786149731490597 us; error 69 us; skew 500 ppm
Aug 08 00:42:11 dist-test-slave-kvfs krb5kdc[20914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:11.493968 20934 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:11.494670 20934 webserver.cc:460] Webserver started at http://127.20.105.126:41009/ using document root <none> and password file <none>
I20260808 00:42:11.495060 20934 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:11.495119 20934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:11.495385 20934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:11.497044 20934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "68cc90d004eb475cb24880e0aa697737"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "0854ad942834db79fef10e7fe3d3406d"
server_key_iv: "80e89a3381fb2c26ecb70af1ae64cac7"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.497814 20934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "68cc90d004eb475cb24880e0aa697737"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "0854ad942834db79fef10e7fe3d3406d"
server_key_iv: "80e89a3381fb2c26ecb70af1ae64cac7"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.499992 20934 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:42:11.501391 20954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:11.501744 20934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:11.502048 20934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "68cc90d004eb475cb24880e0aa697737"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "0854ad942834db79fef10e7fe3d3406d"
server_key_iv: "80e89a3381fb2c26ecb70af1ae64cac7"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.502180 20934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:11.517899 20934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:11.521433 20934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:11.521727 20934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:11.531253 20934 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:32937
I20260808 00:42:11.531344 21032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:32937 every 8 connection(s)
I20260808 00:42:11.532125 20934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:11.533790 21035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:11.537173 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 20934
I20260808 00:42:11.537361 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:11.538623 20901 external_mini_cluster.cc:1428] Setting key 227e87be021ef153d4db2455c9f96a47
I20260808 00:42:11.540340 21035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737: Bootstrap starting.
I20260808 00:42:11.542181 21035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:11.543056 21035 log.cc:824] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:11.544420 21035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737: No bootstrap required, opened a new log
I20260808 00:42:11.548712 21035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cc90d004eb475cb24880e0aa697737" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32937 } }
I20260808 00:42:11.549002 21035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:11.549185 21035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68cc90d004eb475cb24880e0aa697737, State: Initialized, Role: FOLLOWER
I20260808 00:42:11.549431 21035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cc90d004eb475cb24880e0aa697737" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32937 } }
I20260808 00:42:11.549748 21035 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:11.549804 21035 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:11.549907 21035 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:11.551595 21035 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cc90d004eb475cb24880e0aa697737" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32937 } }
I20260808 00:42:11.551858 21035 leader_election.cc:302] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68cc90d004eb475cb24880e0aa697737; no voters:
I20260808 00:42:11.552222 21035 leader_election.cc:288] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:11.552377 21044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:11.552678 21044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [term 1 LEADER]: Becoming Leader. State: Replica: 68cc90d004eb475cb24880e0aa697737, State: Running, Role: LEADER
I20260808 00:42:11.552882 21044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cc90d004eb475cb24880e0aa697737" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32937 } }
I20260808 00:42:11.552922 21035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:11.553840 21045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68cc90d004eb475cb24880e0aa697737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cc90d004eb475cb24880e0aa697737" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32937 } } }
I20260808 00:42:11.553946 21045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:11.554205 21047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68cc90d004eb475cb24880e0aa697737. Latest consensus state: current_term: 1 leader_uuid: "68cc90d004eb475cb24880e0aa697737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68cc90d004eb475cb24880e0aa697737" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32937 } } }
I20260808 00:42:11.554293 21047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:11.554530 21060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:11.555815 21060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:11.559286 21060 catalog_manager.cc:1347] Generated new cluster ID: ee7358ad04154aff851e93d9b6bb692c
I20260808 00:42:11.559352 21060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:42:11 dist-test-slave-kvfs krb5kdc[20914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149731, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:11.563905 21060 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:11.564584 21060 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:11.569211 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:11.540609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39222 (local address 127.20.105.126:32937)
0808 00:42:11.541015 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:11.541029 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:11.541426 (+ 397us) server_negotiation.cc:407] Connection header received
0808 00:42:11.547301 (+ 5875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:11.547311 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:11.547449 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:11.547620 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:11.551030 (+ 3410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.552903 (+ 1873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.555250 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.556112 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.556287 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:11.563209 (+ 6922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:11.563214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:11.563214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:11.563236 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:11.566137 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:11.566870 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:11.566875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:11.566876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:11.567031 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:11.567418 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:11.567419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:11.567419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:11.567778 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:11.567951 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:11.568518 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 00:42:11.568771 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":138,"threads_started":1}
I20260808 00:42:11.573876 21060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68cc90d004eb475cb24880e0aa697737: Generated new TSK 0
I20260808 00:42:11.574118 21060 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:11.611716 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32937
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:11.716339 21075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:11.716682 21075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:11.716729 21075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:11.718147 21075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:11.718230 21075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:11.718284 21075 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:11.720193 21075 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21075
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:11.720605 21075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:11.720909 21075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:11.728785 21087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:11.728778 21085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:11.729012 21075 server_base.cc:1034] running on GCE node
W20260808 00:42:11.728778 21083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:11.729452 21075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:11.729787 21075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:11.730983 21075 hybrid_clock.cc:648] HybridClock initialized: now 1786149731730950 us; error 45 us; skew 500 ppm
Aug 08 00:42:11 dist-test-slave-kvfs krb5kdc[20914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:11.734843 21075 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:11.735462 21075 webserver.cc:460] Webserver started at http://127.20.105.65:35505/ using document root <none> and password file <none>
I20260808 00:42:11.735755 21075 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:11.735806 21075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:11.735993 21075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:11.737346 21075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "1b6c40f0177d4dceb5383376da360fe3"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "b5307d02c888827c061f58af0571a665"
server_key_iv: "35c6373f30f51de82bcccbbb342e0916"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.737774 21075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "1b6c40f0177d4dceb5383376da360fe3"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "b5307d02c888827c061f58af0571a665"
server_key_iv: "35c6373f30f51de82bcccbbb342e0916"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.739584 21075 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:42:11.740768 21096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:11.741089 21075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:11.741206 21075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "1b6c40f0177d4dceb5383376da360fe3"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "b5307d02c888827c061f58af0571a665"
server_key_iv: "35c6373f30f51de82bcccbbb342e0916"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.741300 21075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:11.753480 21075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:11.756561 21075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:11.756753 21075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:11.757293 21075 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:11.757787 21075 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:11.757893 21075 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:11.757964 21075 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:11.757993 21075 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:11.769021 21075 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:35693
I20260808 00:42:11.769120 21273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:35693 every 8 connection(s)
I20260808 00:42:11.769541 21075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:11.770520 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21075
I20260808 00:42:11.770618 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:11.770861 20901 external_mini_cluster.cc:1428] Setting key 9f1a5728e2a2a8562c3572852f5b8c4f
Aug 08 00:42:11 dist-test-slave-kvfs krb5kdc[20914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:11.782356 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:11.770852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:49993 (local address 127.20.105.126:32937)
0808 00:42:11.770944 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:11.770948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:11.771244 (+ 296us) server_negotiation.cc:407] Connection header received
0808 00:42:11.772148 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:11.772150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:11.772164 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:11.772227 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:11.774101 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.774739 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.775906 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.776225 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.776321 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:11.779084 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:11.779086 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:11.779086 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:11.779096 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:11.780737 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:11.781214 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:11.781215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:11.781216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:11.781286 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:11.781578 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:11.781579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:11.781579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:11.781720 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:11.781797 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:11.782132 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 00:42:11.782238 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
I20260808 00:42:11.782608 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:11.770840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:32937 (local address 127.20.105.65:49993)
0808 00:42:11.771060 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:11.771120 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:11.771993 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:11.772314 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:11.772319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:11.773608 (+ 1289us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:11.773964 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:11.773972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.774878 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.774880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:11.775801 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:11.775803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.776334 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.776335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:11.776487 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:11.777158 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:11.777169 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:11.778947 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:11.780848 (+ 1901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:11.780857 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:11.780860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:11.781128 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:11.781367 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:11.781367 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:11.781368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:11.781496 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:11.781798 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:11.781807 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:11.781980 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 00:42:11.782109 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 00:42:11.782306 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":112,"threads_started":1}
I20260808 00:42:11.783418 21277 heartbeater.cc:347] Connected to a master server at 127.20.105.126:32937
I20260808 00:42:11.783595 21277 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:11.783844 21277 heartbeater.cc:511] Master 127.20.105.126:32937 requested a full tablet report, sending...
I20260808 00:42:11.785065 20981 ts_manager.cc:194] Registered new tserver with Master: 1b6c40f0177d4dceb5383376da360fe3 (127.20.105.65:35693)
I20260808 00:42:11.785846 20981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:49993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:11.805433 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32937
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:11.919446 21283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:11.919786 21283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:11.919837 21283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:11.921355 21283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:11.921440 21283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:11.921494 21283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:11.923353 21283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:11.923700 21283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:11.924000 21283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:11.931337 21289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:11.931303 21288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:11.931376 21293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:11.931649 21283 server_base.cc:1034] running on GCE node
I20260808 00:42:11.932080 21283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:11.932421 21283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:11.933619 21283 hybrid_clock.cc:648] HybridClock initialized: now 1786149731933608 us; error 45 us; skew 500 ppm
Aug 08 00:42:11 dist-test-slave-kvfs krb5kdc[20914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:11.936868 21283 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:11.937434 21283 webserver.cc:460] Webserver started at http://127.20.105.66:36845/ using document root <none> and password file <none>
I20260808 00:42:11.937734 21283 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:11.937814 21283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:11.937988 21283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:11.939424 21283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "03d9e4fee93d4380ab131725f378f325"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "511dec37e1366bc4094ee7a4544836c0"
server_key_iv: "e7dab7de466149b5df80f910eadc06c2"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.939955 21283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "03d9e4fee93d4380ab131725f378f325"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "511dec37e1366bc4094ee7a4544836c0"
server_key_iv: "e7dab7de466149b5df80f910eadc06c2"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.941905 21283 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:11.942809 21303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:11.943014 21283 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:11.943142 21283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "03d9e4fee93d4380ab131725f378f325"
format_stamp: "Formatted at 2026-08-08 00:42:11 on dist-test-slave-kvfs"
server_key: "511dec37e1366bc4094ee7a4544836c0"
server_key_iv: "e7dab7de466149b5df80f910eadc06c2"
server_key_version: "encryptionkey@0"
I20260808 00:42:11.943264 21283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:11.962734 21283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:11.965826 21283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:11.966037 21283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:11.966431 21283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:11.966913 21283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:11.966979 21283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:11.967032 21283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:11.967063 21283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:11.977916 21283 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:44459
I20260808 00:42:11.978003 21467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:44459 every 8 connection(s)
I20260808 00:42:11.978489 21283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:11.984140 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21283
I20260808 00:42:11.984272 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:11.984509 20901 external_mini_cluster.cc:1428] Setting key 7b37c61dcb1c41ee2364cd8e7e621cea
Aug 08 00:42:11 dist-test-slave-kvfs krb5kdc[20914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149731, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:11.991746 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:11.979601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:33299 (local address 127.20.105.126:32937)
0808 00:42:11.979727 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:11.979730 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:11.980086 (+ 356us) server_negotiation.cc:407] Connection header received
0808 00:42:11.980869 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:11.980871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:11.980887 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:11.980963 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:11.982248 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.982842 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.984082 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.984372 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.984465 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:11.987350 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:11.987352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:11.987353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:11.987365 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:11.989487 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:11.990088 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:11.990090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:11.990091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:11.990174 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:11.990576 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:11.990577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:11.990578 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:11.990778 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:11.990861 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:11.991442 (+ 581us) server_negotiation.cc:299] Negotiation successful
0808 00:42:11.991577 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 00:42:11.991905 21470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:11.979533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:32937 (local address 127.20.105.66:33299)
0808 00:42:11.979893 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:11.979949 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:11.980736 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:11.981059 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:11.981064 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:11.981806 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:11.982131 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:11.982138 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.983000 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.983001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:11.983943 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:11.983945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:11.984474 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:11.984476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:11.984676 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:11.985359 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:11.985370 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:11.987224 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:11.989611 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:11.989622 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:11.989625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:11.989991 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:11.990308 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:11.990309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:11.990310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:11.990489 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:11.990865 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:11.990871 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:11.991222 (+ 351us) client_negotiation.cc:769] Sending connection context
0808 00:42:11.991383 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 00:42:11.991610 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":179,"threads_started":1}
I20260808 00:42:11.992933 21468 heartbeater.cc:347] Connected to a master server at 127.20.105.126:32937
I20260808 00:42:11.993172 21468 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:11.993520 21468 heartbeater.cc:511] Master 127.20.105.126:32937 requested a full tablet report, sending...
I20260808 00:42:11.994208 20981 ts_manager.cc:194] Registered new tserver with Master: 03d9e4fee93d4380ab131725f378f325 (127.20.105.66:44459)
I20260808 00:42:11.995019 20981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:33299
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:12.024367 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32937
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:12.143477 21476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:12.143761 21476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:12.143790 21476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:12.145252 21476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:12.145303 21476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:12.145334 21476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:12.147039 21476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32937
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:12.147336 21476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:12.147594 21476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:12.154243 21490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:12.154281 21486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:12.154285 21487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:12.154608 21476 server_base.cc:1034] running on GCE node
I20260808 00:42:12.154902 21476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:12.155175 21476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:12.156342 21476 hybrid_clock.cc:648] HybridClock initialized: now 1786149732156296 us; error 54 us; skew 500 ppm
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[20914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:12.159811 21476 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:12.160470 21476 webserver.cc:460] Webserver started at http://127.20.105.67:45595/ using document root <none> and password file <none>
I20260808 00:42:12.160773 21476 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:12.160863 21476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:12.161047 21476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:12.162532 21476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "92aa509a217e4b1ba86bce2309e3e39e"
format_stamp: "Formatted at 2026-08-08 00:42:12 on dist-test-slave-kvfs"
server_key: "191c8c7255ef0604030f4ddd32da1049"
server_key_iv: "8343bc113ff21d8ba0f7f232a6940b22"
server_key_version: "encryptionkey@0"
I20260808 00:42:12.163051 21476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "92aa509a217e4b1ba86bce2309e3e39e"
format_stamp: "Formatted at 2026-08-08 00:42:12 on dist-test-slave-kvfs"
server_key: "191c8c7255ef0604030f4ddd32da1049"
server_key_iv: "8343bc113ff21d8ba0f7f232a6940b22"
server_key_version: "encryptionkey@0"
I20260808 00:42:12.165038 21476 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20260808 00:42:12.166083 21502 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:12.166373 21476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:12.166532 21476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "92aa509a217e4b1ba86bce2309e3e39e"
format_stamp: "Formatted at 2026-08-08 00:42:12 on dist-test-slave-kvfs"
server_key: "191c8c7255ef0604030f4ddd32da1049"
server_key_iv: "8343bc113ff21d8ba0f7f232a6940b22"
server_key_version: "encryptionkey@0"
I20260808 00:42:12.166631 21476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:12.182277 21476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:12.185416 21476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:12.185665 21476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:12.186100 21476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:12.186553 21476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:12.186620 21476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:12.186676 21476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:12.186712 21476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:12.199111 21476 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:35467
I20260808 00:42:12.199164 21670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:35467 every 8 connection(s)
I20260808 00:42:12.199648 21476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:12.203114 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21476
I20260808 00:42:12.203239 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:12.203445 20901 external_mini_cluster.cc:1428] Setting key 3336a6587fc52c2e292567f718f03a63
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[20914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149732, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:12.212255 21038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:12.200765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:60825 (local address 127.20.105.126:32937)
0808 00:42:12.200827 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:12.200829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:12.201169 (+ 340us) server_negotiation.cc:407] Connection header received
0808 00:42:12.201908 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:12.201910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:12.201926 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:12.202007 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:12.203573 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:12.204424 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:12.205902 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:12.206142 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:12.206207 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:12.209130 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:12.209132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:12.209132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:12.209143 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:12.210619 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:12.211054 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:12.211055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:12.211056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:12.211119 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:12.211402 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:12.211403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:12.211403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:12.211584 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:12.211642 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:12.211918 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 00:42:12.212001 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:42:12.212592 21675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:12.200713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:32937 (local address 127.20.105.67:60825)
0808 00:42:12.200992 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:12.201046 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:12.201753 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:12.202108 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:12.202113 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:12.202958 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:12.203417 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:12.203427 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:12.204549 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:12.204550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:12.205796 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:12.205800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:12.206274 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:12.206275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:12.206418 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:12.207129 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:12.207139 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:12.209025 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:12.210704 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:12.210711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:12.210714 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:12.210985 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:12.211213 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:12.211213 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:12.211214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:12.211344 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:12.211645 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:12.211650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:12.211795 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 00:42:12.211893 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 00:42:12.212217 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":142,"threads_started":1}
I20260808 00:42:12.213361 21672 heartbeater.cc:347] Connected to a master server at 127.20.105.126:32937
I20260808 00:42:12.213485 21672 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:12.213681 21672 heartbeater.cc:511] Master 127.20.105.126:32937 requested a full tablet report, sending...
I20260808 00:42:12.214140 20981 ts_manager.cc:194] Registered new tserver with Master: 92aa509a217e4b1ba86bce2309e3e39e (127.20.105.67:35467)
I20260808 00:42:12.214634 20981 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:60825
I20260808 00:42:12.216281 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[20914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149732, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[20914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149732, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:12.253734 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:12.242742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55036 (local address 127.20.105.65:35693)
0808 00:42:12.242944 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:12.242955 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:12.243035 (+ 80us) server_negotiation.cc:407] Connection header received
0808 00:42:12.243196 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:12.243200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:12.243223 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:12.243411 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:12.244248 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:12.245172 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:12.246071 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:12.246547 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:12.246640 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:12.250036 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:12.250050 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:12.250051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:12.250062 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:12.251855 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:12.252399 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:12.252404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:12.252405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:12.252513 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:12.252806 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:12.252808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:12.252808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:12.253193 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:12.253378 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:12.253483 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 00:42:12.253579 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":77,"threads_started":1}
W20260808 00:42:12.254578 21137 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55036
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[20914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[20914](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149732, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:12.287640 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:12.278359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55048 (local address 127.20.105.65:35693)
0808 00:42:12.278450 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:12.278452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:12.278531 (+ 79us) server_negotiation.cc:407] Connection header received
0808 00:42:12.278600 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:12.278602 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:12.278615 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:12.278710 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:12.279501 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:12.280199 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:12.281162 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:12.281392 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:12.281465 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:12.284399 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:12.284401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:12.284402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:12.284413 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:12.285978 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:12.286431 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:12.286432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:12.286433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:12.286506 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:12.286887 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:12.286889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:12.286889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:12.287054 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:12.287159 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:12.287417 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 00:42:12.287525 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 00:42:12.288498 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21075
I20260808 00:42:12.309110 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21283
I20260808 00:42:12.341168 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21476
I20260808 00:42:12.366963 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 20934
2026-08-08T00:42:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2663 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[21693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[21693](info): set up 2 sockets
Aug 08 00:42:12 dist-test-slave-kvfs krb5kdc[21693](info): commencing operation
krb5kdc: starting...
W20260808 00:42:13.693022 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:13 dist-test-slave-kvfs krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:13.853334 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33245
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:33245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:13.953254 21711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:13.953848 21711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:13.953953 21711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:13.955510 21711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:13.955593 21711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:13.955621 21711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:13.955653 21711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:13.955684 21711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:13.957669 21711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:33245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:13.958050 21711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:13.958364 21711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:13.964421 21719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:13.964409 21724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:13.964409 21718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:13.964924 21711 server_base.cc:1034] running on GCE node
I20260808 00:42:13.965199 21711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:13.965507 21711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:13.966734 21711 hybrid_clock.cc:648] HybridClock initialized: now 1786149733966701 us; error 45 us; skew 500 ppm
Aug 08 00:42:13 dist-test-slave-kvfs krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149733, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:13.969903 21711 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:13.970589 21711 webserver.cc:460] Webserver started at http://127.20.105.126:45235/ using document root <none> and password file <none>
I20260808 00:42:13.970885 21711 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:13.970942 21711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:13.971133 21711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:13.972635 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "7cbf06b1c2304002bc526244e362e6cb"
format_stamp: "Formatted at 2026-08-08 00:42:13 on dist-test-slave-kvfs"
server_key: "82396050af341823f8f7949ed63f4102"
server_key_iv: "2fd5c19c9f6e9bd0288f2bf0c5c5490f"
server_key_version: "encryptionkey@0"
I20260808 00:42:13.973095 21711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "7cbf06b1c2304002bc526244e362e6cb"
format_stamp: "Formatted at 2026-08-08 00:42:13 on dist-test-slave-kvfs"
server_key: "82396050af341823f8f7949ed63f4102"
server_key_iv: "2fd5c19c9f6e9bd0288f2bf0c5c5490f"
server_key_version: "encryptionkey@0"
I20260808 00:42:13.974900 21711 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:13.975888 21736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:13.976137 21711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:13.976238 21711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "7cbf06b1c2304002bc526244e362e6cb"
format_stamp: "Formatted at 2026-08-08 00:42:13 on dist-test-slave-kvfs"
server_key: "82396050af341823f8f7949ed63f4102"
server_key_iv: "2fd5c19c9f6e9bd0288f2bf0c5c5490f"
server_key_version: "encryptionkey@0"
I20260808 00:42:13.976300 21711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:13.987564 21711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:13.990340 21711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:13.990489 21711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:13.996325 21711 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:33245
I20260808 00:42:13.996362 21818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:33245 every 8 connection(s)
I20260808 00:42:13.996872 21711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:13.998024 21821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:14.001255 21821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb: Bootstrap starting.
I20260808 00:42:14.001940 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21711
I20260808 00:42:14.002058 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:14.002175 21821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:14.002266 20901 external_mini_cluster.cc:1428] Setting key a8134a7a851e3209d2ddbeb4fc156b28
I20260808 00:42:14.002606 21821 log.cc:824] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:14.003829 21821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb: No bootstrap required, opened a new log
I20260808 00:42:14.006146 21821 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cbf06b1c2304002bc526244e362e6cb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33245 } }
I20260808 00:42:14.006350 21821 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:14.006407 21821 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cbf06b1c2304002bc526244e362e6cb, State: Initialized, Role: FOLLOWER
I20260808 00:42:14.006589 21821 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cbf06b1c2304002bc526244e362e6cb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33245 } }
I20260808 00:42:14.006695 21821 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:14.006743 21821 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:14.006796 21821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:14.007503 21821 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cbf06b1c2304002bc526244e362e6cb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33245 } }
I20260808 00:42:14.007666 21821 leader_election.cc:302] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cbf06b1c2304002bc526244e362e6cb; no voters:
I20260808 00:42:14.007911 21821 leader_election.cc:288] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:14.007957 21828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:14.008301 21828 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [term 1 LEADER]: Becoming Leader. State: Replica: 7cbf06b1c2304002bc526244e362e6cb, State: Running, Role: LEADER
I20260808 00:42:14.008472 21821 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:14.008452 21828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cbf06b1c2304002bc526244e362e6cb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33245 } }
I20260808 00:42:14.009969 21829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cbf06b1c2304002bc526244e362e6cb. Latest consensus state: current_term: 1 leader_uuid: "7cbf06b1c2304002bc526244e362e6cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cbf06b1c2304002bc526244e362e6cb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33245 } } }
I20260808 00:42:14.010066 21829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:14.010227 21833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cbf06b1c2304002bc526244e362e6cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cbf06b1c2304002bc526244e362e6cb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33245 } } }
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:14.010437 21833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb [sys.catalog]: This master's current role is: LEADER
W20260808 00:42:14.010861 21859 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:42:14.011866 21859 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:42:14.012079 21857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:14.012315 21857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:14.014279 21857 catalog_manager.cc:1347] Generated new cluster ID: 9e47c280a25b42b4b880a3226be37603
I20260808 00:42:14.014339 21857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:14.016579 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.003050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34122 (local address 127.20.105.126:33245)
0808 00:42:14.003341 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.003352 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.003385 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:42:14.004110 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.004128 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.004194 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.004367 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:14.005140 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.006159 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.007208 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.007750 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.007909 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.011631 (+ 3722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:14.011638 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:14.011638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:14.011657 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:14.014036 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.014544 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.014548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.014549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.014674 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.014904 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.014905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.014905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.015260 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:14.015467 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.016125 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.016295 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":87,"threads_started":1}
I20260808 00:42:14.032717 21857 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:14.033730 21857 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:14.039093 21857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cbf06b1c2304002bc526244e362e6cb: Generated new TSK 0
I20260808 00:42:14.039415 21857 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:14.083143 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33245
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:14.202929 21866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:14.203236 21866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:14.203269 21866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:14.204727 21866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:14.204782 21866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:14.204813 21866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:14.206614 21866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33245
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:14.206982 21866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:14.207309 21866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:14.214236 21882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:14.214475 21872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:14.214282 21873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:14.214452 21866 server_base.cc:1034] running on GCE node
I20260808 00:42:14.214777 21866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:14.215106 21866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:14.216287 21866 hybrid_clock.cc:648] HybridClock initialized: now 1786149734216268 us; error 55 us; skew 500 ppm
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:14.219434 21866 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:14.220064 21866 webserver.cc:460] Webserver started at http://127.20.105.65:46297/ using document root <none> and password file <none>
I20260808 00:42:14.220378 21866 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:14.220432 21866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:14.220592 21866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:14.221889 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "532cd1024e964f1eb31871bbfcc99808"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "7d15eb5d53b185db7469b149127ed4dc"
server_key_iv: "b28a48b4a57d33e2ab7f987adf7c1a7d"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.222339 21866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "532cd1024e964f1eb31871bbfcc99808"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "7d15eb5d53b185db7469b149127ed4dc"
server_key_iv: "b28a48b4a57d33e2ab7f987adf7c1a7d"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.224242 21866 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:42:14.225334 21894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.225672 21866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:14.225776 21866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "532cd1024e964f1eb31871bbfcc99808"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "7d15eb5d53b185db7469b149127ed4dc"
server_key_iv: "b28a48b4a57d33e2ab7f987adf7c1a7d"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.225941 21866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:14.248899 21866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:14.251677 21866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:14.251880 21866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:14.252341 21866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:14.252780 21866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:14.252843 21866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.252885 21866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:14.252914 21866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.260215 21866 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:40515
I20260808 00:42:14.260273 22072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:40515 every 8 connection(s)
I20260808 00:42:14.260700 21866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:14.263114 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21866
I20260808 00:42:14.263216 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:14.263430 20901 external_mini_cluster.cc:1428] Setting key 573fc177799baff15e439b633854fef6
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:14.273171 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.261659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:56019 (local address 127.20.105.126:33245)
0808 00:42:14.261753 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.261755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.262058 (+ 303us) server_negotiation.cc:407] Connection header received
0808 00:42:14.263368 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.263370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.263388 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.263451 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:14.264545 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.265111 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.266248 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.266479 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.266581 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.269403 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:14.269406 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:14.269406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:14.269416 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:14.271156 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.271816 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.271818 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.271819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.271908 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.272248 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.272249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.272250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.272419 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:14.272529 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.272898 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.273043 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:42:14.273401 22076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.261650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33245 (local address 127.20.105.65:56019)
0808 00:42:14.261870 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:14.261915 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:14.263223 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:14.263546 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:14.263552 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:14.264085 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:14.264430 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.264446 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.265232 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.265233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.266149 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.266152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.266596 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.266597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.266758 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.267563 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:14.267589 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:14.269287 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:14.271376 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:14.271386 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:14.271389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:14.271703 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:14.271988 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:14.271989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:14.271989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:14.272166 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:14.272512 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:14.272522 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:14.272722 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 00:42:14.272859 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 00:42:14.273076 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":132,"threads_started":1}
I20260808 00:42:14.274549 22074 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33245
I20260808 00:42:14.274691 22074 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:14.275028 22074 heartbeater.cc:511] Master 127.20.105.126:33245 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
I20260808 00:42:14.275818 21762 ts_manager.cc:194] Registered new tserver with Master: 532cd1024e964f1eb31871bbfcc99808 (127.20.105.65:40515)
I20260808 00:42:14.276674 21762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:56019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:14.300864 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33245
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:14.407517 22086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:14.407900 22086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:14.407934 22086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:14.409636 22086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:14.409685 22086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:14.409718 22086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:14.412171 22086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33245
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:14.412642 22086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:14.412973 22086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:14.420054 22097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:14.421185 22086 server_base.cc:1034] running on GCE node
W20260808 00:42:14.421286 22099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:14.421701 22094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:14.421996 22086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:14.422369 22086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:14.423575 22086 hybrid_clock.cc:648] HybridClock initialized: now 1786149734423545 us; error 52 us; skew 500 ppm
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:14.427116 22086 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:14.427786 22086 webserver.cc:460] Webserver started at http://127.20.105.66:32895/ using document root <none> and password file <none>
I20260808 00:42:14.428153 22086 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:14.428232 22086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:14.428411 22086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:14.429800 22086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "0f9454b2219d48cf9513638c554f4dff"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "8dbfdbbd4d6af9645f4df2ed9a050f4f"
server_key_iv: "c0e2f31ab21aa40276c3f847bd99b2a6"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.430302 22086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "0f9454b2219d48cf9513638c554f4dff"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "8dbfdbbd4d6af9645f4df2ed9a050f4f"
server_key_iv: "c0e2f31ab21aa40276c3f847bd99b2a6"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.432137 22086 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:42:14.433138 22109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.433359 22086 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:14.433501 22086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "0f9454b2219d48cf9513638c554f4dff"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "8dbfdbbd4d6af9645f4df2ed9a050f4f"
server_key_iv: "c0e2f31ab21aa40276c3f847bd99b2a6"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.433604 22086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:14.448359 22086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:14.451164 22086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:14.451367 22086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:14.451663 22086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:14.452099 22086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:14.452158 22086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.452205 22086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:14.452248 22086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.459892 22086 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:34537
I20260808 00:42:14.459956 22277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:34537 every 8 connection(s)
I20260808 00:42:14.460386 22086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:14.469856 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 22086
I20260808 00:42:14.469977 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:14.470194 20901 external_mini_cluster.cc:1428] Setting key a795f1976740d34e7567d8c7b02f2565
I20260808 00:42:14.473210 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.461301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:49505 (local address 127.20.105.126:33245)
0808 00:42:14.461387 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.461391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.461609 (+ 218us) server_negotiation.cc:407] Connection header received
0808 00:42:14.462810 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.462813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.462828 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.462897 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:14.463680 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.464652 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.465798 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.466163 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.466240 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.469161 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:14.469163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:14.469164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:14.469182 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:14.471267 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.471817 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.471819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.471820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.471903 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.472254 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.472255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.472255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.472444 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:14.472525 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.472917 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.473040 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:42:14.473378 22282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.461249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33245 (local address 127.20.105.66:49505)
0808 00:42:14.461458 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:14.461510 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:14.462666 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:14.463012 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:14.463017 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:14.463322 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:14.463581 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.463585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.464746 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.464747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.465705 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.465708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.466256 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.466257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.466388 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.467066 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:14.467076 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:14.469006 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:14.471371 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:14.471380 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:14.471383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:14.471730 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:14.471986 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:14.471987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:14.471987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:14.472181 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:14.472525 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:14.472533 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:14.472744 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 00:42:14.472875 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:42:14.473081 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":120,"threads_started":1}
I20260808 00:42:14.474397 22278 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33245
I20260808 00:42:14.474553 22278 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:14.474836 22278 heartbeater.cc:511] Master 127.20.105.126:33245 requested a full tablet report, sending...
I20260808 00:42:14.475394 21762 ts_manager.cc:194] Registered new tserver with Master: 0f9454b2219d48cf9513638c554f4dff (127.20.105.66:34537)
I20260808 00:42:14.475970 21762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:49505
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:14.506062 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33245
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:42927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:14.613673 22288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:14.613976 22288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:14.614018 22288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:14.615361 22288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:14.615434 22288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:14.615491 22288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:14.617242 22288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:42927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33245
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:14.617581 22288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:14.617861 22288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:14.624125 22296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:14.624130 22295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:14.624207 22299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:14.624653 22288 server_base.cc:1034] running on GCE node
I20260808 00:42:14.624894 22288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:14.625155 22288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:14.626317 22288 hybrid_clock.cc:648] HybridClock initialized: now 1786149734626304 us; error 38 us; skew 500 ppm
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:14.629734 22288 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:14.630347 22288 webserver.cc:460] Webserver started at http://127.20.105.67:42943/ using document root <none> and password file <none>
I20260808 00:42:14.630628 22288 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:14.630702 22288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:14.630869 22288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:14.632280 22288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "0eb762433a4e404392009012e45dd893"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "246e17fe6abf120d9fd875f1ad8e2ff0"
server_key_iv: "02cc31139288ccf129d6a0440ed69a67"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.632875 22288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "0eb762433a4e404392009012e45dd893"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "246e17fe6abf120d9fd875f1ad8e2ff0"
server_key_iv: "02cc31139288ccf129d6a0440ed69a67"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.634687 22288 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:14.635725 22308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.636147 22288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:14.636298 22288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "0eb762433a4e404392009012e45dd893"
format_stamp: "Formatted at 2026-08-08 00:42:14 on dist-test-slave-kvfs"
server_key: "246e17fe6abf120d9fd875f1ad8e2ff0"
server_key_iv: "02cc31139288ccf129d6a0440ed69a67"
server_key_version: "encryptionkey@0"
I20260808 00:42:14.636412 22288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:14.671741 22288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:14.675184 22288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:14.675434 22288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:14.675771 22288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:14.676287 22288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:14.676362 22288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.676446 22288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:14.676496 22288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:14.685477 22288 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:38901
I20260808 00:42:14.685570 22503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:38901 every 8 connection(s)
I20260808 00:42:14.686093 22288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:14.696040 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 22288
I20260808 00:42:14.696178 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:14.696353 20901 external_mini_cluster.cc:1428] Setting key 0e443dd440953827b5f25fdb87a405da
I20260808 00:42:14.699523 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.687280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:36355 (local address 127.20.105.126:33245)
0808 00:42:14.687366 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.687368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.687642 (+ 274us) server_negotiation.cc:407] Connection header received
0808 00:42:14.688957 (+ 1315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.688959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.688975 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.689050 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:14.690022 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.690698 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.692181 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.692436 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.692526 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.695743 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:14.695747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:14.695748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:14.695783 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:14.697865 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.698351 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.698352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.698352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.698416 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.698747 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.698748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.698748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.698895 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:14.698957 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.699328 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.699426 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:42:14.699710 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.687217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33245 (local address 127.20.105.67:36355)
0808 00:42:14.687425 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:14.687481 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:14.688788 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:14.689195 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:14.689200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:14.689572 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:14.689847 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.689851 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.690832 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.690834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.691981 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.691985 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.692562 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.692564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.692796 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.693586 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:14.693597 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:14.695602 (+ 2005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:14.697964 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:14.697973 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:14.697976 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:14.698278 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:14.698484 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:14.698485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:14.698486 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:14.698677 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:14.698986 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:14.698994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:14.699160 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 00:42:14.699270 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 00:42:14.699447 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
I20260808 00:42:14.700435 22504 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33245
I20260808 00:42:14.700560 22504 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:14.700762 22504 heartbeater.cc:511] Master 127.20.105.126:33245 requested a full tablet report, sending...
I20260808 00:42:14.701220 21762 ts_manager.cc:194] Registered new tserver with Master: 0eb762433a4e404392009012e45dd893 (127.20.105.67:38901)
I20260808 00:42:14.701628 21762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:36355
I20260808 00:42:14.708673 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:42:14.718868 21827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.711413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34124 (local address 127.20.105.126:33245)
0808 00:42:14.711500 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.711502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.711534 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:42:14.711563 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.711566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.711583 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.711652 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:14.712517 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.713098 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.713909 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.714137 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.714228 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.715745 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:14.715747 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:14.715747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:14.715756 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:14.717425 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.717848 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.717849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.717850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.717921 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.718151 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.718152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.718152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.718369 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:14.718501 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.718616 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.718726 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 00:42:14.722443 21762 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:14.724856 21762 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:14.734086 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.728878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:34537 (local address 127.0.0.1:46508)
0808 00:42:14.729160 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:14.729217 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:14.729315 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:14.729743 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:14.729748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:14.729751 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:14.729864 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.729867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.731375 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.731376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.732633 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.732636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.733683 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.733684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.733780 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.733798 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:42:14.733893 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 00:42:14.733963 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":40,"thread_start_us":72,"threads_started":1}
I20260808 00:42:14.734094 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.729214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46508 (local address 127.20.105.66:34537)
0808 00:42:14.729545 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.729550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.729559 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:42:14.729601 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.729604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.729618 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.729718 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:14.729937 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.731223 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.732737 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.733567 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.733657 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.733732 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.733928 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.733980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":137,"threads_started":1}
I20260808 00:42:14.734576 22525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.729217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59094 (local address 127.20.105.67:38901)
0808 00:42:14.729695 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.729699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.729711 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:14.729797 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.729891 (+ 94us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.729919 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.730015 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:14.730268 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.731240 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.733071 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.733858 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.733955 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.734014 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.734403 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.734461 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":126,"threads_started":1}
I20260808 00:42:14.734326 22524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.728898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:38901 (local address 127.0.0.1:59094)
0808 00:42:14.729495 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:14.729506 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:14.729625 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:14.730011 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:14.730012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:14.730014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:14.730065 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.730067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.731454 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.731455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.732963 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.732967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.734052 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.734053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.734124 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.734129 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:14.734184 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:42:14.734238 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":74,"thread_start_us":165,"threads_started":1}
I20260808 00:42:14.734499 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.728898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40515 (local address 127.0.0.1:48120)
0808 00:42:14.729733 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:14.729742 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:14.729853 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:14.730458 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:14.730460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:14.730463 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:14.730584 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.730587 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.731898 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.731899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.733141 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.733144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.734253 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.734254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.734343 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.734348 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:14.734394 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:42:14.734436 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":4,"thread_start_us":167,"threads_started":1}
I20260808 00:42:14.734745 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.729276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48120 (local address 127.20.105.65:40515)
0808 00:42:14.729490 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.729496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.729821 (+ 325us) server_negotiation.cc:407] Connection header received
0808 00:42:14.730311 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.730315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.730335 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.730429 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:14.730676 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.731775 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.733235 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.734073 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.734148 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.734209 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.734574 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.734635 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
I20260808 00:42:14.735812 22177 tablet_service.cc:1467] Processing CreateTablet for tablet ff77731ed7ad483fbc2145c8541eebd2 (DEFAULT_TABLE table=test-table [id=fe7cf796b8344c46b3f6e126501e49c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:14.736371 22177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff77731ed7ad483fbc2145c8541eebd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:14.736480 21974 tablet_service.cc:1467] Processing CreateTablet for tablet ff77731ed7ad483fbc2145c8541eebd2 (DEFAULT_TABLE table=test-table [id=fe7cf796b8344c46b3f6e126501e49c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:14.736825 21974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff77731ed7ad483fbc2145c8541eebd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:14.738480 22390 tablet_service.cc:1467] Processing CreateTablet for tablet ff77731ed7ad483fbc2145c8541eebd2 (DEFAULT_TABLE table=test-table [id=fe7cf796b8344c46b3f6e126501e49c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:14.738981 22390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ff77731ed7ad483fbc2145c8541eebd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:14.740454 22533 tablet_bootstrap.cc:492] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff: Bootstrap starting.
I20260808 00:42:14.741623 22534 tablet_bootstrap.cc:492] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808: Bootstrap starting.
I20260808 00:42:14.742305 22533 tablet_bootstrap.cc:654] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:14.742812 22534 tablet_bootstrap.cc:654] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:14.743614 22533 log.cc:824] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:14.743930 22535 tablet_bootstrap.cc:492] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893: Bootstrap starting.
I20260808 00:42:14.744084 22534 log.cc:824] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:14.744911 22535 tablet_bootstrap.cc:654] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:14.745167 22534 tablet_bootstrap.cc:492] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808: No bootstrap required, opened a new log
I20260808 00:42:14.745232 22534 ts_tablet_manager.cc:1397] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 00:42:14.745595 22535 log.cc:824] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:14.746423 22535 tablet_bootstrap.cc:492] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893: No bootstrap required, opened a new log
I20260808 00:42:14.746513 22535 ts_tablet_manager.cc:1397] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:42:14.747709 22534 raft_consensus.cc:348] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.747877 22534 raft_consensus.cc:374] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:14.747901 22534 raft_consensus.cc:729] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 532cd1024e964f1eb31871bbfcc99808, State: Initialized, Role: FOLLOWER
I20260808 00:42:14.748090 22534 consensus_queue.cc:260] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.748589 22534 ts_tablet_manager.cc:1428] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:42:14.748620 22074 heartbeater.cc:503] Master 127.20.105.126:33245 was elected leader, sending a full tablet report...
I20260808 00:42:14.748473 22535 raft_consensus.cc:348] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.748721 22535 raft_consensus.cc:374] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:14.748771 22535 raft_consensus.cc:729] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0eb762433a4e404392009012e45dd893, State: Initialized, Role: FOLLOWER
I20260808 00:42:14.748893 22535 consensus_queue.cc:260] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.749251 22535 ts_tablet_manager.cc:1428] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:42:14.749323 22504 heartbeater.cc:503] Master 127.20.105.126:33245 was elected leader, sending a full tablet report...
I20260808 00:42:14.749933 22533 tablet_bootstrap.cc:492] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff: No bootstrap required, opened a new log
I20260808 00:42:14.750054 22533 ts_tablet_manager.cc:1397] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20260808 00:42:14.752841 22533 raft_consensus.cc:348] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.753051 22533 raft_consensus.cc:374] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:14.753077 22533 raft_consensus.cc:729] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9454b2219d48cf9513638c554f4dff, State: Initialized, Role: FOLLOWER
I20260808 00:42:14.753223 22533 consensus_queue.cc:260] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.753687 22533 ts_tablet_manager.cc:1428] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260808 00:42:14.753777 22278 heartbeater.cc:503] Master 127.20.105.126:33245 was elected leader, sending a full tablet report...
W20260808 00:42:14.761592 22075 tablet.cc:2369] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:14.778328 22541 raft_consensus.cc:482] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:14.778496 22541 raft_consensus.cc:504] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.779389 22541 leader_election.cc:288] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f9454b2219d48cf9513638c554f4dff (127.20.105.66:34537), 532cd1024e964f1eb31871bbfcc99808 (127.20.105.65:40515)
I20260808 00:42:14.783921 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.779728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:37193 (local address 127.20.105.65:40515)
0808 00:42:14.779793 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.779795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.779807 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:14.779847 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.779848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.779862 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.779924 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:14.780239 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.781160 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.782736 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.783354 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.783449 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.783481 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.783790 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.783838 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:42:14.783931 22508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.779612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40515 (local address 127.20.105.67:37193)
0808 00:42:14.779696 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:14.779714 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:14.779771 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:14.779952 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:14.779953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:14.779956 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:14.780137 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.780141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.781259 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.781260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.782605 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.782609 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.783600 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.783602 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.783709 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.783711 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:42:14.783764 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:42:14.783816 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":9}
I20260808 00:42:14.783921 22528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.779781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:48153 (local address 127.20.105.66:34537)
0808 00:42:14.779841 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.779844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.780153 (+ 309us) server_negotiation.cc:407] Connection header received
0808 00:42:14.780246 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.780248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.780260 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.780333 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:14.780502 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.781344 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.782736 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.783413 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.783481 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.783491 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.783785 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.783834 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:42:14.783941 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.779614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:34537 (local address 127.20.105.67:48153)
0808 00:42:14.780007 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:14.780051 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:14.780150 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:14.780340 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:14.780340 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:14.780342 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:14.780428 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.780430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.781425 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.781426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.782606 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:14.782609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.783601 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.783602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:14.783692 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.783697 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:14.783764 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 00:42:14.783816 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":80,"thread_start_us":163,"threads_started":1}
I20260808 00:42:14.784435 22006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff77731ed7ad483fbc2145c8541eebd2" candidate_uuid: "0eb762433a4e404392009012e45dd893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "532cd1024e964f1eb31871bbfcc99808" is_pre_election: true
I20260808 00:42:14.784693 22006 raft_consensus.cc:2454] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0eb762433a4e404392009012e45dd893 in term 0.
I20260808 00:42:14.784902 22206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff77731ed7ad483fbc2145c8541eebd2" candidate_uuid: "0eb762433a4e404392009012e45dd893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f9454b2219d48cf9513638c554f4dff" is_pre_election: true
I20260808 00:42:14.785128 22206 raft_consensus.cc:2454] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0eb762433a4e404392009012e45dd893 in term 0.
I20260808 00:42:14.785041 22310 leader_election.cc:302] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eb762433a4e404392009012e45dd893, 532cd1024e964f1eb31871bbfcc99808; no voters:
I20260808 00:42:14.785269 22541 raft_consensus.cc:2781] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:14.785359 22541 raft_consensus.cc:482] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:14.785383 22541 raft_consensus.cc:3037] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:14.786356 22541 raft_consensus.cc:504] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.786540 22541 leader_election.cc:288] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [CANDIDATE]: Term 1 election: Requested vote from peers 0f9454b2219d48cf9513638c554f4dff (127.20.105.66:34537), 532cd1024e964f1eb31871bbfcc99808 (127.20.105.65:40515)
I20260808 00:42:14.786686 22206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff77731ed7ad483fbc2145c8541eebd2" candidate_uuid: "0eb762433a4e404392009012e45dd893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f9454b2219d48cf9513638c554f4dff"
I20260808 00:42:14.786698 22006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ff77731ed7ad483fbc2145c8541eebd2" candidate_uuid: "0eb762433a4e404392009012e45dd893" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "532cd1024e964f1eb31871bbfcc99808"
I20260808 00:42:14.786790 22206 raft_consensus.cc:3037] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:14.786787 22006 raft_consensus.cc:3037] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:14.787809 22006 raft_consensus.cc:2454] T ff77731ed7ad483fbc2145c8541eebd2 P 532cd1024e964f1eb31871bbfcc99808 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0eb762433a4e404392009012e45dd893 in term 1.
I20260808 00:42:14.787832 22206 raft_consensus.cc:2454] T ff77731ed7ad483fbc2145c8541eebd2 P 0f9454b2219d48cf9513638c554f4dff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0eb762433a4e404392009012e45dd893 in term 1.
I20260808 00:42:14.788010 22310 leader_election.cc:302] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0eb762433a4e404392009012e45dd893, 532cd1024e964f1eb31871bbfcc99808; no voters:
I20260808 00:42:14.788203 22541 raft_consensus.cc:2781] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:14.788409 22541 raft_consensus.cc:686] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [term 1 LEADER]: Becoming Leader. State: Replica: 0eb762433a4e404392009012e45dd893, State: Running, Role: LEADER
I20260808 00:42:14.788594 22541 consensus_queue.cc:237] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } }
I20260808 00:42:14.789732 21762 catalog_manager.cc:5495] T ff77731ed7ad483fbc2145c8541eebd2 P 0eb762433a4e404392009012e45dd893 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0eb762433a4e404392009012e45dd893 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "0eb762433a4e404392009012e45dd893" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9454b2219d48cf9513638c554f4dff" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0eb762433a4e404392009012e45dd893" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "532cd1024e964f1eb31871bbfcc99808" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40515 } health_report { overall_health: UNKNOWN } } }
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149733, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:14.816100 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.805870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48136 (local address 127.20.105.65:40515)
0808 00:42:14.805982 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.805985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.805997 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:14.806120 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.806123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.806141 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.806214 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:14.806968 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.807528 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.808614 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.808831 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.808928 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.812454 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:14.812476 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:14.812477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:14.812493 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:14.814630 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.815082 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.815088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.815088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.815165 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.815388 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.815389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.815389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.815596 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:14.815784 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.815825 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.815929 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:14.855752 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.845211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48140 (local address 127.20.105.65:40515)
0808 00:42:14.845302 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.845305 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.845508 (+ 203us) server_negotiation.cc:407] Connection header received
0808 00:42:14.845646 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.845647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.845662 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.845735 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:14.846553 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.847273 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.848789 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.849027 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.849106 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.852539 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:14.852541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:14.852542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:14.852553 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:14.854245 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.854703 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.854705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.854705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.854783 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.855007 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.855008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.855008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.855150 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:14.855268 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.855504 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.855617 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:42:14 dist-test-slave-kvfs krb5kdc[21693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:14.886791 22532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:14.877905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48146 (local address 127.20.105.65:40515)
0808 00:42:14.878013 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:14.878016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:14.878143 (+ 127us) server_negotiation.cc:407] Connection header received
0808 00:42:14.878262 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:14.878264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:14.878284 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:14.878361 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:14.879103 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.879757 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.880748 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:14.880960 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:14.881043 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:14.883895 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:14.883896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:14.883897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:14.883905 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:14.885389 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.885773 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.885774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.885775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.885840 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:14.886139 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:14.886140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:14.886141 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:14.886268 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:14.886373 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:14.886564 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 00:42:14.886645 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 00:42:14.887683 21935 tablet_service.cc:2908] Rejecting scan request for tablet ff77731ed7ad483fbc2145c8541eebd2: Uninitialized: clean time has not yet been initialized
I20260808 00:42:14.889127 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21866
I20260808 00:42:14.906687 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 22086
I20260808 00:42:14.922219 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 22288
I20260808 00:42:14.938505 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21711
2026-08-08T00:42:14Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2576 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:15 dist-test-slave-kvfs krb5kdc[22571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:15 dist-test-slave-kvfs krb5kdc[22571](info): set up 2 sockets
Aug 08 00:42:15 dist-test-slave-kvfs krb5kdc[22571](info): commencing operation
krb5kdc: starting...
W20260808 00:42:16.242627 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:16 dist-test-slave-kvfs krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:16.396793 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:41293
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:41293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:16.501124 22587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:16.501488 22587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:16.501533 22587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:16.502931 22587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:16.503019 22587 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:42:16.503053 22587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:16.503078 22587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:16.503113 22587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:16.503137 22587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:16.505010 22587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:41293
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:41293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:16.505378 22587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:16.505669 22587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:16.511490 22594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:16.511516 22592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:16.511761 22587 server_base.cc:1034] running on GCE node
W20260808 00:42:16.511490 22597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:16.512197 22587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:16.512580 22587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:16.513760 22587 hybrid_clock.cc:648] HybridClock initialized: now 1786149736513735 us; error 48 us; skew 500 ppm
Aug 08 00:42:16 dist-test-slave-kvfs krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149736, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:16.517145 22587 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:16.517833 22587 webserver.cc:460] Webserver started at http://127.20.105.126:34361/ using document root <none> and password file <none>
I20260808 00:42:16.518131 22587 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:16.518205 22587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:16.518378 22587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:16.519704 22587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "0333deef3de14c269c553b2195e7029f"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "750d3a7d2c4830d30f8c88d03e08b686"
server_key_iv: "009046ce1586f9e8bc3f083ae1b7811c"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.520200 22587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "0333deef3de14c269c553b2195e7029f"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "750d3a7d2c4830d30f8c88d03e08b686"
server_key_iv: "009046ce1586f9e8bc3f083ae1b7811c"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.521802 22587 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:16.522715 22612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:16.522946 22587 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:16.523068 22587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "0333deef3de14c269c553b2195e7029f"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "750d3a7d2c4830d30f8c88d03e08b686"
server_key_iv: "009046ce1586f9e8bc3f083ae1b7811c"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.523165 22587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:16.537447 22587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:16.539912 22587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:16.540136 22587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:16.546754 22587 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:41293
I20260808 00:42:16.546844 22706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:41293 every 8 connection(s)
I20260808 00:42:16.547235 22587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:16.548064 22707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:16.550966 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f: Bootstrap starting.
I20260808 00:42:16.551736 22707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:16.552155 22707 log.cc:824] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:16.553108 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f: No bootstrap required, opened a new log
I20260808 00:42:16.555631 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 22587
I20260808 00:42:16.555738 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:16.556000 20901 external_mini_cluster.cc:1428] Setting key 5f27105706621af925a6a2fa14229cac
I20260808 00:42:16.555843 22707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0333deef3de14c269c553b2195e7029f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41293 } }
I20260808 00:42:16.556126 22707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:16.556205 22707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0333deef3de14c269c553b2195e7029f, State: Initialized, Role: FOLLOWER
I20260808 00:42:16.556392 22707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0333deef3de14c269c553b2195e7029f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41293 } }
I20260808 00:42:16.556488 22707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:16.556536 22707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:16.556599 22707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:16.557396 22707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0333deef3de14c269c553b2195e7029f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41293 } }
I20260808 00:42:16.557536 22707 leader_election.cc:302] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0333deef3de14c269c553b2195e7029f; no voters:
I20260808 00:42:16.557761 22707 leader_election.cc:288] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:16.557812 22713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:16.558017 22713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [term 1 LEADER]: Becoming Leader. State: Replica: 0333deef3de14c269c553b2195e7029f, State: Running, Role: LEADER
I20260808 00:42:16.558200 22713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0333deef3de14c269c553b2195e7029f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41293 } }
I20260808 00:42:16.558197 22707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:16.558669 22718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0333deef3de14c269c553b2195e7029f. Latest consensus state: current_term: 1 leader_uuid: "0333deef3de14c269c553b2195e7029f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0333deef3de14c269c553b2195e7029f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41293 } } }
I20260808 00:42:16.558926 22718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:16.558856 22714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0333deef3de14c269c553b2195e7029f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0333deef3de14c269c553b2195e7029f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41293 } } }
I20260808 00:42:16.559190 22714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:16.559428 22720 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:16.560804 22720 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:16.563148 22720 catalog_manager.cc:1347] Generated new cluster ID: 411e4ebe4847442b8935aa5f81ae75a8
I20260808 00:42:16.563207 22720 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:42:16 dist-test-slave-kvfs krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:16.570304 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.556740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.20.105.126:41293)
0808 00:42:16.556987 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:16.557005 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:16.557038 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:42:16.557162 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:16.557169 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:16.557223 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:16.557386 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:16.558329 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.559810 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.561136 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.561748 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.561858 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:16.565134 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:16.565139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:16.565139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:16.565153 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:16.568191 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:16.568640 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:16.568643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:16.568644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:16.568776 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:16.568970 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:16.568971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:16.568971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:16.569298 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:16.569459 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:16.569912 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 00:42:16.570057 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":116,"threads_started":1}
I20260808 00:42:16.597909 22720 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:16.598588 22720 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:16.612290 22720 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f: Generated new TSK 0
I20260808 00:42:16.612619 22720 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:16.652074 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41293
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:16.755628 22752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:16.756049 22752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:16.756136 22752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:16.757651 22752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:16.757740 22752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:16.757777 22752 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:16.757822 22752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:16.759708 22752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:41293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:16.760140 22752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:16.760469 22752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:16.766590 22761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:16.766600 22762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:16.766613 22767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:16.766925 22752 server_base.cc:1034] running on GCE node
I20260808 00:42:16.767254 22752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:16.767557 22752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:16.768743 22752 hybrid_clock.cc:648] HybridClock initialized: now 1786149736768737 us; error 41 us; skew 500 ppm
Aug 08 00:42:16 dist-test-slave-kvfs krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149736, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:16.771648 22752 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:16.772264 22752 webserver.cc:460] Webserver started at http://127.20.105.65:43865/ using document root <none> and password file <none>
I20260808 00:42:16.772529 22752 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:16.772575 22752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:16.772683 22752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:16.774012 22752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "c0df6e1ab0fd49168f48465b22a074d0"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "a37bab4bb0cc6b8e8add18b0cd941967"
server_key_iv: "5563d693da2c3c80b17acf1dbb3a0af6"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.774533 22752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "c0df6e1ab0fd49168f48465b22a074d0"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "a37bab4bb0cc6b8e8add18b0cd941967"
server_key_iv: "5563d693da2c3c80b17acf1dbb3a0af6"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.776304 22752 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:42:16.777220 22781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:16.777453 22752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:16.777560 22752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "c0df6e1ab0fd49168f48465b22a074d0"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "a37bab4bb0cc6b8e8add18b0cd941967"
server_key_iv: "5563d693da2c3c80b17acf1dbb3a0af6"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.777626 22752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:16.785857 22752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:16.788702 22752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:16.788836 22752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:16.789381 22752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:16.789419 22752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:16.789443 22752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:16.789455 22752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:42:16 dist-test-slave-kvfs krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149736, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:16.798408 22752 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:34729
I20260808 00:42:16.798527 22985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:34729 every 8 connection(s)
I20260808 00:42:16.798979 22752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:16.800936 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 22752
I20260808 00:42:16.801097 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:16.801323 20901 external_mini_cluster.cc:1428] Setting key 895181619ae641a4a0f7329ae7be334d
I20260808 00:42:16.802057 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.790198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:54753 (local address 127.20.105.126:41293)
0808 00:42:16.790292 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:16.790295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:16.790491 (+ 196us) server_negotiation.cc:407] Connection header received
0808 00:42:16.791197 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:16.791199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:16.791216 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:16.791298 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:16.792362 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.793333 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.794563 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.795012 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.795134 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:16.798277 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:16.798279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:16.798280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:16.798295 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:16.800176 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:16.800693 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:16.800695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:16.800696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:16.800793 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:16.801156 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:16.801157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:16.801157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:16.801344 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:16.801426 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:16.801865 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 00:42:16.801977 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:42:16.802347 22795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.790104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41293 (local address 127.20.105.65:54753)
0808 00:42:16.790334 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:16.790388 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:16.791057 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:16.791409 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:16.791414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:16.791814 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:16.792233 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:16.792241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.793472 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.793473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:16.794474 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:16.794477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.795145 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.795146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:16.795337 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:16.796084 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:16.796096 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:16.798155 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:16.800261 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:16.800269 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:16.800272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:16.800552 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:16.800910 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:16.800912 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:16.800912 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:16.801044 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:16.801431 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:16.801439 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:16.801670 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 00:42:16.801832 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 00:42:16.802065 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":110,"threads_started":1}
I20260808 00:42:16.802999 22986 heartbeater.cc:347] Connected to a master server at 127.20.105.126:41293
I20260808 00:42:16.803130 22986 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:16.803360 22986 heartbeater.cc:511] Master 127.20.105.126:41293 requested a full tablet report, sending...
I20260808 00:42:16.804134 22632 ts_manager.cc:194] Registered new tserver with Master: c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729)
I20260808 00:42:16.805100 22632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:54753
I20260808 00:42:16.811604 22994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.804590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41293 (local address 127.20.105.65:45369)
0808 00:42:16.804751 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:16.804765 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:16.804835 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:16.804982 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:16.804983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:16.805264 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:16.805395 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:16.805398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.806193 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.806194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:16.806860 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:16.806863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.807298 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.807299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:16.807398 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:16.808004 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:16.808010 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:16.808518 (+ 508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:16.810382 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:16.810383 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:16.810384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:16.810597 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:16.810834 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:16.810835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:16.810836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:16.810910 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:16.811201 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:16.811203 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:16.811273 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 00:42:16.811358 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 00:42:16.811486 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260808 00:42:16.811594 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.804652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:45369 (local address 127.20.105.126:41293)
0808 00:42:16.804691 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:16.804693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:16.804833 (+ 140us) server_negotiation.cc:407] Connection header received
0808 00:42:16.804901 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:16.804902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:16.804912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:16.804975 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:16.805492 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.806076 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.806973 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.807216 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.807300 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:16.808643 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:16.808645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:16.808646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:16.808654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:16.810253 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:16.810686 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:16.810687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:16.810688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:16.810756 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:16.810987 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:16.810988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:16.810988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:16.811124 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:16.811202 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:16.811385 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 00:42:16.811483 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:16.841109 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41293
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:16.944011 22998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:16.944362 22998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:16.944397 22998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:16.945837 22998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:16.945884 22998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:16.945907 22998 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:16.945926 22998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:16.947638 22998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:41293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:16.947911 22998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:16.948212 22998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:16.954903 23013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:16.955180 22998 server_base.cc:1034] running on GCE node
W20260808 00:42:16.954905 23007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:16.954909 23006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:16.955570 22998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:16.955929 22998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:16.957118 22998 hybrid_clock.cc:648] HybridClock initialized: now 1786149736957073 us; error 62 us; skew 500 ppm
Aug 08 00:42:16 dist-test-slave-kvfs krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149736, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:16.961014 22998 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:16.961716 22998 webserver.cc:460] Webserver started at http://127.20.105.66:38467/ using document root <none> and password file <none>
I20260808 00:42:16.962066 22998 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:16.962152 22998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:16.962347 22998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:16.963932 22998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "4b13f5867ffa437ea05712055902ee78"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "5da031bfac1cb964b07103f95e3d1dca"
server_key_iv: "d26f1cc5bc8b87e1257c6fed1347a3d1"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.964449 22998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "4b13f5867ffa437ea05712055902ee78"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "5da031bfac1cb964b07103f95e3d1dca"
server_key_iv: "d26f1cc5bc8b87e1257c6fed1347a3d1"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.966150 22998 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:16.967111 23025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:16.967353 22998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:16.967483 22998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "4b13f5867ffa437ea05712055902ee78"
format_stamp: "Formatted at 2026-08-08 00:42:16 on dist-test-slave-kvfs"
server_key: "5da031bfac1cb964b07103f95e3d1dca"
server_key_iv: "d26f1cc5bc8b87e1257c6fed1347a3d1"
server_key_version: "encryptionkey@0"
I20260808 00:42:16.967593 22998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:16.978686 22998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:16.981469 22998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:16.981667 22998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:16.982489 22998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:16.982538 22998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:16.982566 22998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:16.982578 22998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:42:16 dist-test-slave-kvfs krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149736, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:16.991506 22998 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:34411
I20260808 00:42:16.992166 22998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:16.992430 23209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:34411 every 8 connection(s)
I20260808 00:42:16.996291 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.983003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:34431 (local address 127.20.105.126:41293)
0808 00:42:16.983087 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:16.983090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:16.983393 (+ 303us) server_negotiation.cc:407] Connection header received
0808 00:42:16.984105 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:16.984107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:16.984125 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:16.984179 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:16.984973 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.985557 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.986796 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.987027 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.987124 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:16.992084 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:16.992087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:16.992088 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:16.992101 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:16.994016 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:16.994683 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:16.994684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:16.994685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:16.994787 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:16.995209 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:16.995210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:16.995210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:16.995396 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:16.995481 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:16.995985 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 00:42:16.996201 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 00:42:16.996568 23039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.983021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41293 (local address 127.20.105.66:34431)
0808 00:42:16.983225 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:16.983274 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:16.983944 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:16.984300 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:16.984307 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:16.984613 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:16.984851 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:16.984856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.985685 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.985686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:16.986687 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:16.986690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:16.987138 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:16.987140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:16.987286 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:16.987962 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:16.987972 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:16.991886 (+ 3914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:16.994145 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:16.994154 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:16.994157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:16.994578 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:16.994956 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:16.994957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:16.994958 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:16.995112 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:16.995508 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:16.995516 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:16.995787 (+ 271us) client_negotiation.cc:769] Sending connection context
0808 00:42:16.995950 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 00:42:16.996253 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260808 00:42:16.997325 23210 heartbeater.cc:347] Connected to a master server at 127.20.105.126:41293
I20260808 00:42:16.997699 23210 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:16.998071 23210 heartbeater.cc:511] Master 127.20.105.126:41293 requested a full tablet report, sending...
I20260808 00:42:16.998648 22632 ts_manager.cc:194] Registered new tserver with Master: 4b13f5867ffa437ea05712055902ee78 (127.20.105.66:34411)
I20260808 00:42:16.999253 22632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:34431
I20260808 00:42:17.000996 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 22998
I20260808 00:42:17.001133 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:17.001427 20901 external_mini_cluster.cc:1428] Setting key 778a1b958636934e9a5b29d3741737e0
I20260808 00:42:17.006943 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.999251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:41751 (local address 127.20.105.126:41293)
0808 00:42:16.999297 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:16.999299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:16.999452 (+ 153us) server_negotiation.cc:407] Connection header received
0808 00:42:16.999541 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:16.999542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:16.999554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:16.999612 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:17.000250 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.000815 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.001683 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.002009 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.002102 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.003415 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:17.003418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:17.003418 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:17.003428 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:17.005563 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.006044 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.006045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.006046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.006109 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.006331 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.006332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.006332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.006523 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:17.006581 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.006767 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.006853 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:42:17.007400 23220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:16.999172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41293 (local address 127.20.105.66:41751)
0808 00:42:16.999374 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:16.999387 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:16.999472 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:16.999617 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:16.999618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:16.999938 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:17.000131 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.000134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.000936 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.000937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.001589 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.001592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.002112 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.002114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.002210 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.002859 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:17.002866 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:17.003286 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:17.005669 (+ 2383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:17.005671 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:17.005671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:17.005958 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:17.006180 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:17.006181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:17.006181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:17.006266 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:17.006591 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:17.006592 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:17.006655 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.006739 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.006881 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:17.036000 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41293
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:32909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:17.136596 23223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:17.136910 23223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:17.136955 23223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:17.138312 23223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:17.138406 23223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:17.138443 23223 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:17.138484 23223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:17.140200 23223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:32909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:41293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:17.140545 23223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:17.140846 23223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:17.146800 23231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:17.146944 23233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:17.146982 23235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:17.147394 23223 server_base.cc:1034] running on GCE node
I20260808 00:42:17.147679 23223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:17.147980 23223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:17.149200 23223 hybrid_clock.cc:648] HybridClock initialized: now 1786149737149175 us; error 44 us; skew 500 ppm
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149737, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:17.152415 23223 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:17.152985 23223 webserver.cc:460] Webserver started at http://127.20.105.67:35175/ using document root <none> and password file <none>
I20260808 00:42:17.153264 23223 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:17.153343 23223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:17.153525 23223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:17.154915 23223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "5785a26ddced4b02bf73f4b1343d8758"
format_stamp: "Formatted at 2026-08-08 00:42:17 on dist-test-slave-kvfs"
server_key: "cb8d1f1467abbdfa908ce12a88d836f0"
server_key_iv: "d85b7f2d4c494af5a6492f883de8296c"
server_key_version: "encryptionkey@0"
I20260808 00:42:17.155385 23223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "5785a26ddced4b02bf73f4b1343d8758"
format_stamp: "Formatted at 2026-08-08 00:42:17 on dist-test-slave-kvfs"
server_key: "cb8d1f1467abbdfa908ce12a88d836f0"
server_key_iv: "d85b7f2d4c494af5a6492f883de8296c"
server_key_version: "encryptionkey@0"
I20260808 00:42:17.157145 23223 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 00:42:17.158104 23247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:17.158295 23223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:17.158402 23223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "5785a26ddced4b02bf73f4b1343d8758"
format_stamp: "Formatted at 2026-08-08 00:42:17 on dist-test-slave-kvfs"
server_key: "cb8d1f1467abbdfa908ce12a88d836f0"
server_key_iv: "d85b7f2d4c494af5a6492f883de8296c"
server_key_version: "encryptionkey@0"
I20260808 00:42:17.158469 23223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:17.184621 23223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:17.187525 23223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:17.187680 23223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:17.188380 23223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:17.188428 23223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:17.188457 23223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:17.188469 23223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149737, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:17.197770 23223 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:44905
I20260808 00:42:17.197888 23419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:44905 every 8 connection(s)
I20260808 00:42:17.198341 23223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:17.201418 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.188964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:53659 (local address 127.20.105.126:41293)
0808 00:42:17.189063 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.189065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.189367 (+ 302us) server_negotiation.cc:407] Connection header received
0808 00:42:17.190310 (+ 943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.190312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.190326 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.190415 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:17.191164 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.191726 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.192962 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.193181 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.193265 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.198061 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:17.198064 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:17.198065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:17.198078 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:17.199780 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.200250 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.200251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.200252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.200320 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.200571 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.200572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.200573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.200716 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:17.200776 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.201230 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.201332 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:42:17.201515 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.188941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41293 (local address 127.20.105.67:53659)
0808 00:42:17.189179 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.189239 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.190137 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.190517 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.190522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.190810 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:17.191058 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.191063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.191828 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.191829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.192841 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.192844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.193271 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.193272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.193420 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.194212 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:17.194227 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:17.197856 (+ 3629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:17.199874 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:17.199881 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:17.199883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:17.200175 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:17.200387 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:17.200388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:17.200388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:17.200506 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:17.200852 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:17.200859 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:17.201025 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.201137 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.201305 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":119,"threads_started":1}
I20260808 00:42:17.202217 23420 heartbeater.cc:347] Connected to a master server at 127.20.105.126:41293
I20260808 00:42:17.202365 23420 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:17.202598 23420 heartbeater.cc:511] Master 127.20.105.126:41293 requested a full tablet report, sending...
I20260808 00:42:17.203117 22633 ts_manager.cc:194] Registered new tserver with Master: 5785a26ddced4b02bf73f4b1343d8758 (127.20.105.67:44905)
I20260808 00:42:17.203651 22633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:53659
I20260808 00:42:17.205008 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 23223
I20260808 00:42:17.205154 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:17.205401 20901 external_mini_cluster.cc:1428] Setting key e1a7353e4d8197d0baa6cb00a2f21cda
I20260808 00:42:17.206652 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:42:17.210775 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.203606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:36805 (local address 127.20.105.126:41293)
0808 00:42:17.203659 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.203661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.203811 (+ 150us) server_negotiation.cc:407] Connection header received
0808 00:42:17.203881 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.203881 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.203893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.204006 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:17.204421 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.205012 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.206128 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.206381 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.206470 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.207688 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:17.207689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:17.207690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:17.207697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:17.209350 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.209829 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.209831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.209831 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.209899 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.210157 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.210158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.210158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.210291 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:17.210378 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.210562 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.210659 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:42:17.210790 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.203549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41293 (local address 127.20.105.67:36805)
0808 00:42:17.203729 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.203739 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.203827 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.204013 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.204047 (+ 34us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.204257 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:17.204336 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.204338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.205150 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.205152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.206009 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.206012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.206467 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.206469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.206575 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.207187 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:17.207192 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:17.207548 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:17.209495 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:17.209496 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:17.209497 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:17.209727 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:17.209988 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:17.209988 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:17.209989 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:17.210065 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:17.210366 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:17.210367 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:17.210431 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.210532 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.210663 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[22571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:17.237984 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.228965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.20.105.126:41293)
0808 00:42:17.229065 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.229068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.229077 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:42:17.229111 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.229112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.229123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.229183 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:17.230004 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.230685 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.231598 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.231809 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.231873 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.234853 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:17.234855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:17.234856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:17.234868 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:17.236618 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.237075 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.237077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.237077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.237148 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.237354 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.237355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.237355 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.237527 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:17.237628 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.237794 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.237885 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 00:42:17.240406 22633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:17.242800 22633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:17.253400 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.247125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44905 (local address 127.0.0.1:49826)
0808 00:42:17.247601 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.247663 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.247766 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.248801 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.248804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.248807 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:17.248915 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.248917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.250823 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.250824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.251909 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.251912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.253012 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.253014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.253112 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.253130 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.253213 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.253283 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":34,"thread_start_us":142,"threads_started":1}
I20260808 00:42:17.253551 23449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.248416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:34729 (local address 127.0.0.1:60964)
0808 00:42:17.248590 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.248601 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.248666 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.249810 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.249812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.249814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:17.249864 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.249866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.251298 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.251299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.252409 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.252412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.253313 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.253314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.253376 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.253379 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.253440 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.253481 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":114,"threads_started":1}
I20260808 00:42:17.253659 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.247643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44264 (local address 127.20.105.66:34411)
0808 00:42:17.248223 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.248230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.248738 (+ 508us) server_negotiation.cc:407] Connection header received
0808 00:42:17.248795 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.248798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.248822 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.249060 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:17.249205 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.250658 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.252281 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.253118 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.253190 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.253245 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.253467 (+ 222us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.253515 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":141,"threads_started":1}
I20260808 00:42:17.253717 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.248081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:34411 (local address 127.0.0.1:44264)
0808 00:42:17.248584 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.248597 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.248666 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.249056 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.249057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.249060 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:17.249136 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.249138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.250752 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.250753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.252170 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.252173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.253194 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.253195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.253271 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.253277 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.253334 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.253380 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":42,"thread_start_us":110,"threads_started":1}
I20260808 00:42:17.253995 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.247476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.20.105.67:44905)
0808 00:42:17.248568 (+ 1092us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.248573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.248586 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:42:17.248638 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.248641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.248660 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.248995 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:17.249090 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.250684 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.252050 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.252911 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.253004 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.253081 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.253828 (+ 747us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.253876 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1047,"thread_start_us":475,"threads_started":1}
I20260808 00:42:17.254720 23319 tablet_service.cc:1467] Processing CreateTablet for tablet d9004e281dbb485597114bfcb0822831 (DEFAULT_TABLE table=test-table [id=3cc21c5f9e9d41beb263d3aa2344dd1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:17.254722 23110 tablet_service.cc:1467] Processing CreateTablet for tablet d9004e281dbb485597114bfcb0822831 (DEFAULT_TABLE table=test-table [id=3cc21c5f9e9d41beb263d3aa2344dd1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:17.255214 23110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9004e281dbb485597114bfcb0822831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.255227 23319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9004e281dbb485597114bfcb0822831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.259719 23455 tablet_bootstrap.cc:492] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: Bootstrap starting.
I20260808 00:42:17.259999 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.249360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.20.105.65:34729)
0808 00:42:17.249604 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.249609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.249623 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:42:17.249670 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.249672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.249690 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.249934 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:17.250019 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.251202 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.252506 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.253237 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.253299 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.253355 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.253456 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.259851 (+ 6395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":123,"threads_started":1}
I20260808 00:42:17.260080 23454 tablet_bootstrap.cc:492] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: Bootstrap starting.
I20260808 00:42:17.260838 23455 tablet_bootstrap.cc:654] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.260838 22874 tablet_service.cc:1467] Processing CreateTablet for tablet d9004e281dbb485597114bfcb0822831 (DEFAULT_TABLE table=test-table [id=3cc21c5f9e9d41beb263d3aa2344dd1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:17.261230 23454 tablet_bootstrap.cc:654] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.261261 22874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9004e281dbb485597114bfcb0822831. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.261283 23455 log.cc:824] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:17.261658 23454 log.cc:824] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:17.262370 23455 tablet_bootstrap.cc:492] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: No bootstrap required, opened a new log
I20260808 00:42:17.262454 23455 ts_tablet_manager.cc:1397] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:42:17.262667 23454 tablet_bootstrap.cc:492] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: No bootstrap required, opened a new log
I20260808 00:42:17.262737 23454 ts_tablet_manager.cc:1397] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:42:17.264919 23455 raft_consensus.cc:348] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.265132 23455 raft_consensus.cc:374] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.265064 23454 raft_consensus.cc:348] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.265172 23455 raft_consensus.cc:729] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b13f5867ffa437ea05712055902ee78, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.265209 23454 raft_consensus.cc:374] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.265250 23454 raft_consensus.cc:729] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5785a26ddced4b02bf73f4b1343d8758, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.265312 23455 consensus_queue.cc:260] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.265421 23454 consensus_queue.cc:260] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.265663 23455 ts_tablet_manager.cc:1428] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:42:17.265723 23454 ts_tablet_manager.cc:1428] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:42:17.265818 23210 heartbeater.cc:503] Master 127.20.105.126:41293 was elected leader, sending a full tablet report...
I20260808 00:42:17.265837 23420 heartbeater.cc:503] Master 127.20.105.126:41293 was elected leader, sending a full tablet report...
I20260808 00:42:17.265998 23463 tablet_bootstrap.cc:492] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Bootstrap starting.
I20260808 00:42:17.267066 23463 tablet_bootstrap.cc:654] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.267434 23463 log.cc:824] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:17.268245 23463 tablet_bootstrap.cc:492] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: No bootstrap required, opened a new log
I20260808 00:42:17.268302 23463 ts_tablet_manager.cc:1397] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:17.270054 23463 raft_consensus.cc:348] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.270208 23463 raft_consensus.cc:374] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.270232 23463 raft_consensus.cc:729] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0df6e1ab0fd49168f48465b22a074d0, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.270359 23463 consensus_queue.cc:260] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.270727 23463 ts_tablet_manager.cc:1428] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:42:17.270816 22986 heartbeater.cc:503] Master 127.20.105.126:41293 was elected leader, sending a full tablet report...
W20260808 00:42:17.300192 22987 tablet.cc:2369] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:17.404084 23467 raft_consensus.cc:482] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:17.404345 23467 raft_consensus.cc:504] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.404858 23467 leader_election.cc:288] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b13f5867ffa437ea05712055902ee78 (127.20.105.66:34411), c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729)
I20260808 00:42:17.408277 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.405302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:34729 (local address 127.20.105.67:36427)
0808 00:42:17.405539 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.405552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.405623 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.405778 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.405779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.405780 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:17.405855 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.405857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.406555 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.406556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.407379 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.407380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.407972 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.407973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.408091 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.408096 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.408149 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.408183 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":3,"thread_start_us":144,"threads_started":1}
I20260808 00:42:17.408370 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.405441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:36427 (local address 127.20.105.65:34729)
0808 00:42:17.405501 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.405504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.405639 (+ 135us) server_negotiation.cc:407] Connection header received
0808 00:42:17.405680 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.405682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.405694 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.405753 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:17.405931 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.406454 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.407483 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.407879 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.407939 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.407949 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.408187 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.408260 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:42:17.408473 23259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.405150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:34411 (local address 127.20.105.67:43723)
0808 00:42:17.405231 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.405246 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.405305 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.405563 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.405564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.405566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:17.405681 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.405683 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.406487 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.406488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.407380 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.407382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.408012 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.408013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.408281 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.408285 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.408351 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.408399 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":37}
I20260808 00:42:17.408666 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.405262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:43723 (local address 127.20.105.66:34411)
0808 00:42:17.405336 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.405339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.405354 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:42:17.405403 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.405404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.405422 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.405525 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:17.405782 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.406391 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.407493 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.407895 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.408000 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.408011 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.408531 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.408586 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:42:17.408797 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9004e281dbb485597114bfcb0822831" candidate_uuid: "5785a26ddced4b02bf73f4b1343d8758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0df6e1ab0fd49168f48465b22a074d0" is_pre_election: true
I20260808 00:42:17.409031 22907 raft_consensus.cc:2454] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5785a26ddced4b02bf73f4b1343d8758 in term 0.
I20260808 00:42:17.409070 23139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9004e281dbb485597114bfcb0822831" candidate_uuid: "5785a26ddced4b02bf73f4b1343d8758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b13f5867ffa437ea05712055902ee78" is_pre_election: true
I20260808 00:42:17.409291 23139 raft_consensus.cc:2454] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5785a26ddced4b02bf73f4b1343d8758 in term 0.
I20260808 00:42:17.409332 23252 leader_election.cc:302] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5785a26ddced4b02bf73f4b1343d8758, c0df6e1ab0fd49168f48465b22a074d0; no voters:
I20260808 00:42:17.409500 23467 raft_consensus.cc:2781] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:17.409567 23467 raft_consensus.cc:482] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:17.409596 23467 raft_consensus.cc:3037] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.410787 23467 raft_consensus.cc:504] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.410949 23467 leader_election.cc:288] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [CANDIDATE]: Term 1 election: Requested vote from peers 4b13f5867ffa437ea05712055902ee78 (127.20.105.66:34411), c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729)
I20260808 00:42:17.411176 23139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9004e281dbb485597114bfcb0822831" candidate_uuid: "5785a26ddced4b02bf73f4b1343d8758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b13f5867ffa437ea05712055902ee78"
I20260808 00:42:17.411221 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9004e281dbb485597114bfcb0822831" candidate_uuid: "5785a26ddced4b02bf73f4b1343d8758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0df6e1ab0fd49168f48465b22a074d0"
I20260808 00:42:17.411309 23139 raft_consensus.cc:3037] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.411304 22907 raft_consensus.cc:3037] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.412348 22907 raft_consensus.cc:2454] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5785a26ddced4b02bf73f4b1343d8758 in term 1.
I20260808 00:42:17.412523 23252 leader_election.cc:302] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5785a26ddced4b02bf73f4b1343d8758, c0df6e1ab0fd49168f48465b22a074d0; no voters:
I20260808 00:42:17.412609 23139 raft_consensus.cc:2454] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5785a26ddced4b02bf73f4b1343d8758 in term 1.
I20260808 00:42:17.412631 23467 raft_consensus.cc:2781] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:17.412809 23467 raft_consensus.cc:686] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 LEADER]: Becoming Leader. State: Replica: 5785a26ddced4b02bf73f4b1343d8758, State: Running, Role: LEADER
I20260808 00:42:17.412932 23467 consensus_queue.cc:237] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } }
I20260808 00:42:17.414083 22633 catalog_manager.cc:5495] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5785a26ddced4b02bf73f4b1343d8758 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "5785a26ddced4b02bf73f4b1343d8758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:17.440405 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.437400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49836 (local address 127.20.105.67:44905)
0808 00:42:17.437478 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.437481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.437493 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:17.437622 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.437623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.437646 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.437729 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:17.437931 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.438490 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.439369 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.439590 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.439680 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.439924 (+ 244us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:17.440086 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:17.440190 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.440233 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.440258 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:42:17.443354 22907 raft_consensus.cc:1261] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Refusing update from remote peer 5785a26ddced4b02bf73f4b1343d8758: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:17.443351 23139 raft_consensus.cc:1261] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Refusing update from remote peer 5785a26ddced4b02bf73f4b1343d8758: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:17.443940 23473 consensus_queue.cc:1035] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:17.444132 23467 consensus_queue.cc:1035] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:17.453193 22632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:17.453424 22632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:17.454772 22632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f: Sending DeleteTablet for 3 replicas of tablet d9004e281dbb485597114bfcb0822831
I20260808 00:42:17.455464 23319 tablet_service.cc:1514] Processing DeleteTablet for tablet d9004e281dbb485597114bfcb0822831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:49826
I20260808 00:42:17.455493 23110 tablet_service.cc:1514] Processing DeleteTablet for tablet d9004e281dbb485597114bfcb0822831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:44264
I20260808 00:42:17.455803 23498 tablet_replica.cc:330] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: stopping tablet replica
I20260808 00:42:17.455924 23498 raft_consensus.cc:2229] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:17.455928 22874 tablet_service.cc:1514] Processing DeleteTablet for tablet d9004e281dbb485597114bfcb0822831 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:60964
I20260808 00:42:17.456136 23498 raft_consensus.cc:2258] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:17.456221 22632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:17.456416 22632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:17.456584 23500 tablet_replica.cc:330] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: stopping tablet replica
I20260808 00:42:17.456760 23498 ts_tablet_manager.cc:1905] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:17.456799 23500 raft_consensus.cc:2229] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:17.457103 23499 tablet_replica.cc:330] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: stopping tablet replica
I20260808 00:42:17.457248 23499 raft_consensus.cc:2229] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:17.457381 23499 raft_consensus.cc:2258] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:17.458055 23500 raft_consensus.cc:2258] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:17.458288 22874 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc8358e7523455fa7e67a445769b2c1 (DEFAULT_TABLE table=test-table [id=8d8add8192ff4c63bae4ac78be008d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:17.458436 22874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc8358e7523455fa7e67a445769b2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.458964 23500 ts_tablet_manager.cc:1905] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:17.459021 23498 ts_tablet_manager.cc:1918] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:17.459095 23498 log.cc:1196] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/d9004e281dbb485597114bfcb0822831
I20260808 00:42:17.459385 23498 ts_tablet_manager.cc:1939] T d9004e281dbb485597114bfcb0822831 P 5785a26ddced4b02bf73f4b1343d8758: Deleting consensus metadata
I20260808 00:42:17.459872 22613 catalog_manager.cc:4841] TS 5785a26ddced4b02bf73f4b1343d8758 (127.20.105.67:44905): tablet d9004e281dbb485597114bfcb0822831 (table test-table [id=3cc21c5f9e9d41beb263d3aa2344dd1b]) successfully deleted
I20260808 00:42:17.460177 23110 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc8358e7523455fa7e67a445769b2c1 (DEFAULT_TABLE table=test-table [id=8d8add8192ff4c63bae4ac78be008d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:17.460338 23110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc8358e7523455fa7e67a445769b2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.461357 23500 ts_tablet_manager.cc:1918] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:17.461475 23500 log.cc:1196] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/d9004e281dbb485597114bfcb0822831
I20260808 00:42:17.461866 23500 ts_tablet_manager.cc:1939] T d9004e281dbb485597114bfcb0822831 P c0df6e1ab0fd49168f48465b22a074d0: Deleting consensus metadata
I20260808 00:42:17.462205 23319 tablet_service.cc:1467] Processing CreateTablet for tablet 5cc8358e7523455fa7e67a445769b2c1 (DEFAULT_TABLE table=test-table [id=8d8add8192ff4c63bae4ac78be008d85]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:17.462342 23319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cc8358e7523455fa7e67a445769b2c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.462476 23455 tablet_bootstrap.cc:492] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: Bootstrap starting.
I20260808 00:42:17.463057 23455 tablet_bootstrap.cc:654] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.463887 23455 tablet_bootstrap.cc:492] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: No bootstrap required, opened a new log
I20260808 00:42:17.463929 23454 tablet_bootstrap.cc:492] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: Bootstrap starting.
I20260808 00:42:17.463949 23455 ts_tablet_manager.cc:1397] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:17.464087 22617 catalog_manager.cc:4841] TS c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729): tablet d9004e281dbb485597114bfcb0822831 (table test-table [id=3cc21c5f9e9d41beb263d3aa2344dd1b]) successfully deleted
I20260808 00:42:17.464155 23455 raft_consensus.cc:348] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.464284 23455 raft_consensus.cc:374] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.464313 23455 raft_consensus.cc:729] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b13f5867ffa437ea05712055902ee78, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.464361 23455 consensus_queue.cc:260] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.464481 23455 ts_tablet_manager.cc:1428] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:17.464607 23463 tablet_bootstrap.cc:492] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: Bootstrap starting.
I20260808 00:42:17.464658 23454 tablet_bootstrap.cc:654] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.465204 23463 tablet_bootstrap.cc:654] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.465718 23499 ts_tablet_manager.cc:1905] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:17.466250 23463 tablet_bootstrap.cc:492] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: No bootstrap required, opened a new log
I20260808 00:42:17.466318 23463 ts_tablet_manager.cc:1397] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:42:17.466497 23463 raft_consensus.cc:348] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.466604 23463 raft_consensus.cc:374] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.466632 23463 raft_consensus.cc:729] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0df6e1ab0fd49168f48465b22a074d0, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.466683 23463 consensus_queue.cc:260] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.466782 23463 ts_tablet_manager.cc:1428] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:42:17.467903 23499 ts_tablet_manager.cc:1918] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:17.467989 23499 log.cc:1196] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/d9004e281dbb485597114bfcb0822831
I20260808 00:42:17.468319 23499 ts_tablet_manager.cc:1939] T d9004e281dbb485597114bfcb0822831 P 4b13f5867ffa437ea05712055902ee78: Deleting consensus metadata
I20260808 00:42:17.468916 22617 catalog_manager.cc:4841] TS 4b13f5867ffa437ea05712055902ee78 (127.20.105.66:34411): tablet d9004e281dbb485597114bfcb0822831 (table test-table [id=3cc21c5f9e9d41beb263d3aa2344dd1b]) successfully deleted
I20260808 00:42:17.469209 23454 tablet_bootstrap.cc:492] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: No bootstrap required, opened a new log
I20260808 00:42:17.469285 23454 ts_tablet_manager.cc:1397] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 00:42:17.469499 23454 raft_consensus.cc:348] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.469586 23454 raft_consensus.cc:374] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.469609 23454 raft_consensus.cc:729] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5785a26ddced4b02bf73f4b1343d8758, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.469659 23454 consensus_queue.cc:260] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.469756 23454 ts_tablet_manager.cc:1428] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260808 00:42:17.493134 23211 tablet.cc:2369] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:17.543095 23491 raft_consensus.cc:482] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:17.543254 23491 raft_consensus.cc:504] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.543542 23491 leader_election.cc:288] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729), 4b13f5867ffa437ea05712055902ee78 (127.20.105.66:34411)
I20260808 00:42:17.543888 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc8358e7523455fa7e67a445769b2c1" candidate_uuid: "5785a26ddced4b02bf73f4b1343d8758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0df6e1ab0fd49168f48465b22a074d0" is_pre_election: true
I20260808 00:42:17.543888 23139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc8358e7523455fa7e67a445769b2c1" candidate_uuid: "5785a26ddced4b02bf73f4b1343d8758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b13f5867ffa437ea05712055902ee78" is_pre_election: true
I20260808 00:42:17.544062 22907 raft_consensus.cc:2454] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5785a26ddced4b02bf73f4b1343d8758 in term 0.
I20260808 00:42:17.544075 23139 raft_consensus.cc:2454] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5785a26ddced4b02bf73f4b1343d8758 in term 0.
I20260808 00:42:17.544291 23252 leader_election.cc:302] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b13f5867ffa437ea05712055902ee78, 5785a26ddced4b02bf73f4b1343d8758; no voters:
I20260808 00:42:17.544431 23491 raft_consensus.cc:2781] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:17.544483 23491 raft_consensus.cc:482] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:17.544507 23491 raft_consensus.cc:3037] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.545482 23491 raft_consensus.cc:504] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.545658 23491 leader_election.cc:288] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [CANDIDATE]: Term 1 election: Requested vote from peers c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729), 4b13f5867ffa437ea05712055902ee78 (127.20.105.66:34411)
I20260808 00:42:17.545905 23139 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc8358e7523455fa7e67a445769b2c1" candidate_uuid: "5785a26ddced4b02bf73f4b1343d8758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b13f5867ffa437ea05712055902ee78"
I20260808 00:42:17.545939 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cc8358e7523455fa7e67a445769b2c1" candidate_uuid: "5785a26ddced4b02bf73f4b1343d8758" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0df6e1ab0fd49168f48465b22a074d0"
I20260808 00:42:17.546034 23139 raft_consensus.cc:3037] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.546054 22907 raft_consensus.cc:3037] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.547250 23139 raft_consensus.cc:2454] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5785a26ddced4b02bf73f4b1343d8758 in term 1.
I20260808 00:42:17.547519 23252 leader_election.cc:302] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b13f5867ffa437ea05712055902ee78, 5785a26ddced4b02bf73f4b1343d8758; no voters:
I20260808 00:42:17.547766 22907 raft_consensus.cc:2454] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5785a26ddced4b02bf73f4b1343d8758 in term 1.
I20260808 00:42:17.547771 23491 raft_consensus.cc:2781] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:17.547843 23491 raft_consensus.cc:686] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 LEADER]: Becoming Leader. State: Replica: 5785a26ddced4b02bf73f4b1343d8758, State: Running, Role: LEADER
I20260808 00:42:17.547914 23491 consensus_queue.cc:237] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.548863 22632 catalog_manager.cc:5495] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5785a26ddced4b02bf73f4b1343d8758 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "5785a26ddced4b02bf73f4b1343d8758" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } health_report { overall_health: HEALTHY } } }
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149736, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:17.595120 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.585143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41293 (local address 127.0.0.1:51066)
0808 00:42:17.585366 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.585378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.585461 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.585695 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.585696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.586109 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:17.586173 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.586176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.587274 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.587275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.587966 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.587968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.588728 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.588730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.588787 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.589560 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:17.589566 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:17.591334 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:17.593574 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:17.593588 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:17.593592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:17.593874 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:17.594184 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:17.594185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:17.594185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:17.594266 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:17.594677 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:17.594692 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:17.594771 (+ 79us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.594846 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.594998 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":119,"threads_started":1}
I20260808 00:42:17.595259 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.585252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51066 (local address 127.20.105.126:41293)
0808 00:42:17.585358 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.585360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.585462 (+ 102us) server_negotiation.cc:407] Connection header received
0808 00:42:17.585526 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.585527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.585544 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.585644 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:17.586309 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.587155 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.588346 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.588631 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.588746 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.591455 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:17.591457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:17.591458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:17.591468 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:17.593456 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.593976 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.593978 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.593978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.594062 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.594343 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.594344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.594344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.594534 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:17.594608 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.594894 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.595129 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":108,"server-negotiator.queue_time_us":57}
I20260808 00:42:17.598824 22632 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:51066:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:42:17.599285 22632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:17.601380 23319 tablet_service.cc:1467] Processing CreateTablet for tablet 28288e5dbcb84f21bd13cfa04a2b46f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76045c176f3a4166be71f42030420b7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:17.601704 23319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28288e5dbcb84f21bd13cfa04a2b46f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.601387 23110 tablet_service.cc:1467] Processing CreateTablet for tablet 28288e5dbcb84f21bd13cfa04a2b46f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76045c176f3a4166be71f42030420b7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:17.603235 23110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28288e5dbcb84f21bd13cfa04a2b46f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.603605 23454 tablet_bootstrap.cc:492] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: Bootstrap starting.
I20260808 00:42:17.604244 23454 tablet_bootstrap.cc:654] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.605801 23455 tablet_bootstrap.cc:492] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: Bootstrap starting.
I20260808 00:42:17.606662 23455 tablet_bootstrap.cc:654] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.607975 23455 tablet_bootstrap.cc:492] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: No bootstrap required, opened a new log
I20260808 00:42:17.608179 23455 ts_tablet_manager.cc:1397] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:17.608421 23455 raft_consensus.cc:348] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.608541 23455 raft_consensus.cc:374] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.608562 23455 raft_consensus.cc:729] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b13f5867ffa437ea05712055902ee78, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.608398 22874 tablet_service.cc:1467] Processing CreateTablet for tablet 28288e5dbcb84f21bd13cfa04a2b46f6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=76045c176f3a4166be71f42030420b7b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:17.608613 23455 consensus_queue.cc:260] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.608731 23455 ts_tablet_manager.cc:1428] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:17.608742 22874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28288e5dbcb84f21bd13cfa04a2b46f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:17.608891 23465 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.608980 23465 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:17.611214 23463 tablet_bootstrap.cc:492] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: Bootstrap starting.
I20260808 00:42:17.611723 23454 tablet_bootstrap.cc:492] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: No bootstrap required, opened a new log
I20260808 00:42:17.611819 23454 ts_tablet_manager.cc:1397] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 00:42:17.612001 23463 tablet_bootstrap.cc:654] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:17.612015 23454 raft_consensus.cc:348] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.612224 23454 raft_consensus.cc:374] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.612291 23454 raft_consensus.cc:729] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5785a26ddced4b02bf73f4b1343d8758, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.612385 23454 consensus_queue.cc:260] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.612545 23454 ts_tablet_manager.cc:1428] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:17.612689 23467 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.612785 23467 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:17.613101 23463 tablet_bootstrap.cc:492] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: No bootstrap required, opened a new log
I20260808 00:42:17.613150 23463 ts_tablet_manager.cc:1397] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:17.613360 23463 raft_consensus.cc:348] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.613451 23463 raft_consensus.cc:374] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:17.613476 23463 raft_consensus.cc:729] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0df6e1ab0fd49168f48465b22a074d0, State: Initialized, Role: FOLLOWER
I20260808 00:42:17.613526 23463 consensus_queue.cc:260] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.613631 23463 ts_tablet_manager.cc:1428] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:17.613858 23470 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.614084 23470 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:17.647146 23465 raft_consensus.cc:482] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:17.647328 23465 raft_consensus.cc:504] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.647902 23465 leader_election.cc:288] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729), 5785a26ddced4b02bf73f4b1343d8758 (127.20.105.67:44905)
I20260808 00:42:17.653226 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.648363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:59431 (local address 127.20.105.65:34729)
0808 00:42:17.648421 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.648423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.648653 (+ 230us) server_negotiation.cc:407] Connection header received
0808 00:42:17.648692 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.648694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.648705 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.648776 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:17.649205 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.650047 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.651947 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.652571 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.652660 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.652672 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.653072 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.653135 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:42:17.653225 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.648328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:39677 (local address 127.20.105.67:44905)
0808 00:42:17.648397 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.648400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.648588 (+ 188us) server_negotiation.cc:407] Connection header received
0808 00:42:17.648676 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.648678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.648697 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.648776 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:17.649231 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.650046 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.651936 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.652580 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.652660 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.652672 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.653070 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.653133 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:42:17.653347 23522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.648235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44905 (local address 127.20.105.66:39677)
0808 00:42:17.648461 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.648473 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.648584 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.648795 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.648795 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.648797 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:17.649136 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.649138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.650148 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.650149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.651812 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.651817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.652820 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.652821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.652953 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.652957 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.653016 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.653084 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":93,"thread_start_us":131,"threads_started":1}
I20260808 00:42:17.653635 23521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.648259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:34729 (local address 127.20.105.66:59431)
0808 00:42:17.648559 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.648571 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.648636 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.648793 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.648794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.648796 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:17.649043 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.649046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.650149 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.650150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.651813 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.651817 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.652785 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.652788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.652931 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.652940 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.653016 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.653084 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":87,"thread_start_us":105,"threads_started":1}
I20260808 00:42:17.653725 23351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28288e5dbcb84f21bd13cfa04a2b46f6" candidate_uuid: "4b13f5867ffa437ea05712055902ee78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5785a26ddced4b02bf73f4b1343d8758" is_pre_election: true
I20260808 00:42:17.654021 23351 raft_consensus.cc:2454] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b13f5867ffa437ea05712055902ee78 in term 0.
I20260808 00:42:17.653980 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28288e5dbcb84f21bd13cfa04a2b46f6" candidate_uuid: "4b13f5867ffa437ea05712055902ee78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0df6e1ab0fd49168f48465b22a074d0" is_pre_election: true
I20260808 00:42:17.654114 22907 raft_consensus.cc:2454] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b13f5867ffa437ea05712055902ee78 in term 0.
I20260808 00:42:17.654363 23027 leader_election.cc:302] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b13f5867ffa437ea05712055902ee78, 5785a26ddced4b02bf73f4b1343d8758; no voters:
I20260808 00:42:17.654529 23465 raft_consensus.cc:2781] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:17.654577 23465 raft_consensus.cc:482] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:17.654610 23465 raft_consensus.cc:3037] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.655786 23465 raft_consensus.cc:504] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.655954 23465 leader_election.cc:288] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [CANDIDATE]: Term 1 election: Requested vote from peers c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729), 5785a26ddced4b02bf73f4b1343d8758 (127.20.105.67:44905)
I20260808 00:42:17.656160 22907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28288e5dbcb84f21bd13cfa04a2b46f6" candidate_uuid: "4b13f5867ffa437ea05712055902ee78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0df6e1ab0fd49168f48465b22a074d0"
I20260808 00:42:17.656255 22907 raft_consensus.cc:3037] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.657220 22907 raft_consensus.cc:2454] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b13f5867ffa437ea05712055902ee78 in term 1.
I20260808 00:42:17.657392 23351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28288e5dbcb84f21bd13cfa04a2b46f6" candidate_uuid: "4b13f5867ffa437ea05712055902ee78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5785a26ddced4b02bf73f4b1343d8758"
I20260808 00:42:17.657451 23032 leader_election.cc:302] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b13f5867ffa437ea05712055902ee78, c0df6e1ab0fd49168f48465b22a074d0; no voters:
I20260808 00:42:17.657495 23351 raft_consensus.cc:3037] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:17.657567 23465 raft_consensus.cc:2781] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:17.657784 23465 raft_consensus.cc:686] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [term 1 LEADER]: Becoming Leader. State: Replica: 4b13f5867ffa437ea05712055902ee78, State: Running, Role: LEADER
I20260808 00:42:17.658020 23465 consensus_queue.cc:237] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } }
I20260808 00:42:17.658543 23351 raft_consensus.cc:2454] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b13f5867ffa437ea05712055902ee78 in term 1.
I20260808 00:42:17.658579 23523 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: TxnStatusTablet state changed. Reason: New leader 4b13f5867ffa437ea05712055902ee78. Latest consensus state: current_term: 1 leader_uuid: "4b13f5867ffa437ea05712055902ee78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.658708 23523 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:17.658964 23525 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:42:17.659116 23525 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 00:42:17.659240 22632 catalog_manager.cc:5495] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b13f5867ffa437ea05712055902ee78 (127.20.105.66). New cstate: current_term: 1 leader_uuid: "4b13f5867ffa437ea05712055902ee78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:17.676644 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.672610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:34411 (local address 127.0.0.1:44270)
0808 00:42:17.672721 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.672736 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.672832 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.673059 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.673061 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.673063 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:17.673170 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.673175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.673975 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.673977 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.675169 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.675174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.675705 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.675707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.675861 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.675956 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:17.676433 (+ 477us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:17.676447 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.676495 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.676564 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 00:42:17.676651 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.672644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.20.105.66:34411)
0808 00:42:17.672718 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.672721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.672778 (+ 57us) server_negotiation.cc:407] Connection header received
0808 00:42:17.672948 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.672948 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.672970 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.673032 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:17.673296 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.673863 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.675299 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.675515 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.675587 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.676143 (+ 556us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:17.676352 (+ 209us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:17.676445 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.676502 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.676538 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:42:17.679236 22907 raft_consensus.cc:1261] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Refusing update from remote peer 4b13f5867ffa437ea05712055902ee78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:17.679370 23351 raft_consensus.cc:1261] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 FOLLOWER]: Refusing update from remote peer 4b13f5867ffa437ea05712055902ee78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:17.679629 23465 consensus_queue.cc:1035] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:17.679836 23523 consensus_queue.cc:1035] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:17.680697 23467 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: TxnStatusTablet state changed. Reason: New leader 4b13f5867ffa437ea05712055902ee78. Latest consensus state: current_term: 1 leader_uuid: "4b13f5867ffa437ea05712055902ee78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.680811 23467 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:17.681059 23470 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: TxnStatusTablet state changed. Reason: New leader 4b13f5867ffa437ea05712055902ee78. Latest consensus state: current_term: 1 leader_uuid: "4b13f5867ffa437ea05712055902ee78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.681178 23470 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:17.681696 23523 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b13f5867ffa437ea05712055902ee78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.681787 23523 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:17.681865 23485 mvcc.cc:205] Tried to move back new op lower bound from 7316069325531205632 to 7316069325448212480. Current Snapshot: MvccSnapshot[applied={T|T < 7316069325531205632 or (T in {7316069325531205632})}]
I20260808 00:42:17.682070 23483 mvcc.cc:205] Tried to move back new op lower bound from 7316069325531205632 to 7316069325448212480. Current Snapshot: MvccSnapshot[applied={T|T < 7316069325531205632}]
I20260808 00:42:17.682368 23467 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b13f5867ffa437ea05712055902ee78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.682457 23467 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 5785a26ddced4b02bf73f4b1343d8758: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:17.683378 23465 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4b13f5867ffa437ea05712055902ee78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.683476 23465 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P 4b13f5867ffa437ea05712055902ee78: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:17.683826 23484 mvcc.cc:205] Tried to move back new op lower bound from 7316069325531205632 to 7316069325448212480. Current Snapshot: MvccSnapshot[applied={T|T < 7316069325531205632}]
I20260808 00:42:17.683964 23470 tablet_replica.cc:439] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4b13f5867ffa437ea05712055902ee78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 } } peers { permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 } } peers { permanent_uuid: "5785a26ddced4b02bf73f4b1343d8758" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44905 } } }
I20260808 00:42:17.684096 23470 tablet_replica.cc:442] T 28288e5dbcb84f21bd13cfa04a2b46f6 P c0df6e1ab0fd49168f48465b22a074d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:17.690065 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.687266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:57643 (local address 127.20.105.66:34411)
0808 00:42:17.687320 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.687323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.687377 (+ 54us) server_negotiation.cc:407] Connection header received
0808 00:42:17.687445 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.687446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.687456 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.687538 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:17.687723 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.688335 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.689213 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.689433 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.689518 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.689745 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:17.689793 (+ 48us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:17.689869 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.689943 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.689972 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:42:17.690099 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.687189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:34411 (local address 127.20.105.67:57643)
0808 00:42:17.687287 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.687303 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.687396 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.687542 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.687544 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.687546 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:17.687631 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.687634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.688438 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.688440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.689094 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.689097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.689534 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.689536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.689614 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.689634 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:17.689871 (+ 237us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:17.689874 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.689937 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.690014 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:42:17.693275 23139 raft_consensus.cc:1261] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Refusing update from remote peer 5785a26ddced4b02bf73f4b1343d8758: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:17.693368 22907 raft_consensus.cc:1261] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Refusing update from remote peer 5785a26ddced4b02bf73f4b1343d8758: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:17.693594 23467 consensus_queue.cc:1035] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b13f5867ffa437ea05712055902ee78" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:17.693722 23491 consensus_queue.cc:1035] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0df6e1ab0fd49168f48465b22a074d0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:17.704610 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.701009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:57985 (local address 127.20.105.67:44905)
0808 00:42:17.701058 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.701060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.701265 (+ 205us) server_negotiation.cc:407] Connection header received
0808 00:42:17.701326 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.701327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.701339 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.701402 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:17.701634 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.702268 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.703653 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.703936 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.704021 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.704307 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:17.704352 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:17.704408 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.704487 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.704513 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:42:17.704727 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.700957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44905 (local address 127.20.105.66:57985)
0808 00:42:17.701161 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:17.701176 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:17.701268 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:17.701404 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:17.701405 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:17.701406 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:17.701553 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.701555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.702352 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.702354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.703564 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:17.703568 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.704042 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.704043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:17.704182 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.704204 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:17.704420 (+ 216us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:17.704422 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:42:17.704482 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:42:17.704529 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":102,"threads_started":1}
I20260808 00:42:17.713784 22632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:17.713955 22632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:17.714833 22632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0333deef3de14c269c553b2195e7029f: Sending DeleteTablet for 3 replicas of tablet 5cc8358e7523455fa7e67a445769b2c1
I20260808 00:42:17.715184 22874 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc8358e7523455fa7e67a445769b2c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:60964
I20260808 00:42:17.715234 23110 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc8358e7523455fa7e67a445769b2c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:44264
I20260808 00:42:17.715243 23319 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cc8358e7523455fa7e67a445769b2c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:49826
I20260808 00:42:17.715309 23500 tablet_replica.cc:330] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: stopping tablet replica
I20260808 00:42:17.715327 23499 tablet_replica.cc:330] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: stopping tablet replica
I20260808 00:42:17.715350 23498 tablet_replica.cc:330] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: stopping tablet replica
I20260808 00:42:17.715370 23500 raft_consensus.cc:2229] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:17.715373 23499 raft_consensus.cc:2229] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:17.715408 23498 raft_consensus.cc:2229] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:17.715413 23500 raft_consensus.cc:2258] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:17.715413 23499 raft_consensus.cc:2258] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:17.715479 23498 raft_consensus.cc:2258] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:17.716178 23499 ts_tablet_manager.cc:1905] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:17.718595 23499 ts_tablet_manager.cc:1918] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:17.718683 23499 log.cc:1196] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/5cc8358e7523455fa7e67a445769b2c1
I20260808 00:42:17.718950 23499 ts_tablet_manager.cc:1939] T 5cc8358e7523455fa7e67a445769b2c1 P 4b13f5867ffa437ea05712055902ee78: Deleting consensus metadata
I20260808 00:42:17.719398 22617 catalog_manager.cc:4841] TS 4b13f5867ffa437ea05712055902ee78 (127.20.105.66:34411): tablet 5cc8358e7523455fa7e67a445769b2c1 (table test-table [id=8d8add8192ff4c63bae4ac78be008d85]) successfully deleted
I20260808 00:42:17.719457 23498 ts_tablet_manager.cc:1905] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:17.721191 23500 ts_tablet_manager.cc:1905] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:17.721300 23498 ts_tablet_manager.cc:1918] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:17.721359 23498 log.cc:1196] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/5cc8358e7523455fa7e67a445769b2c1
I20260808 00:42:17.721593 23498 ts_tablet_manager.cc:1939] T 5cc8358e7523455fa7e67a445769b2c1 P 5785a26ddced4b02bf73f4b1343d8758: Deleting consensus metadata
I20260808 00:42:17.722044 22613 catalog_manager.cc:4841] TS 5785a26ddced4b02bf73f4b1343d8758 (127.20.105.67:44905): tablet 5cc8358e7523455fa7e67a445769b2c1 (table test-table [id=8d8add8192ff4c63bae4ac78be008d85]) successfully deleted
I20260808 00:42:17.723215 23500 ts_tablet_manager.cc:1918] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:17.723264 23500 log.cc:1196] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/5cc8358e7523455fa7e67a445769b2c1
I20260808 00:42:17.723433 23500 ts_tablet_manager.cc:1939] T 5cc8358e7523455fa7e67a445769b2c1 P c0df6e1ab0fd49168f48465b22a074d0: Deleting consensus metadata
I20260808 00:42:17.724051 22617 catalog_manager.cc:4841] TS c0df6e1ab0fd49168f48465b22a074d0 (127.20.105.65:34729): tablet 5cc8358e7523455fa7e67a445769b2c1 (table test-table [id=8d8add8192ff4c63bae4ac78be008d85]) successfully deleted
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[22571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:17.733505 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.722274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60978 (local address 127.20.105.65:34729)
0808 00:42:17.722327 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.722329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.722611 (+ 282us) server_negotiation.cc:407] Connection header received
0808 00:42:17.722673 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.722674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.722694 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.722765 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:17.723729 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.724431 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.725538 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.725725 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.725812 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.730049 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:17.730065 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:17.730065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:17.730082 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:17.732121 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.732518 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.732524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.732524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.732610 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.732825 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.732826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.732827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.733022 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:17.733207 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.733276 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.733374 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 00:42:17.734020 22976 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60978
I20260808 00:42:17.742389 22712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:17.735398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51068 (local address 127.20.105.126:41293)
0808 00:42:17.735480 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:17.735482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:17.735525 (+ 43us) server_negotiation.cc:407] Connection header received
0808 00:42:17.735589 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:17.735590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:17.735604 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:17.735673 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:17.736230 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.737091 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.737811 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:17.738166 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:17.738268 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:17.739458 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:17.739460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:17.739460 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:17.739469 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:17.741241 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.741583 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.741584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.741585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.741658 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:17.741844 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:17.741845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:17.741845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:17.742004 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:17.742118 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:17.742185 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 00:42:17.742270 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260808 00:42:17.742717 22632 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51068
I20260808 00:42:17.743731 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 22752
I20260808 00:42:17.753240 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 22998
I20260808 00:42:17.772224 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 23223
I20260808 00:42:17.790429 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 22587
2026-08-08T00:42:17Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2851 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[23556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[23556](info): set up 2 sockets
Aug 08 00:42:17 dist-test-slave-kvfs krb5kdc[23556](info): commencing operation
krb5kdc: starting...
W20260808 00:42:19.082233 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:19 dist-test-slave-kvfs krb5kdc[23556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:19.240296 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34001
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:34001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:19.345518 23581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:19.345849 23581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:19.345892 23581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:19.347167 23581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:19.347254 23581 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:42:19.347276 23581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:19.347301 23581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:19.347327 23581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:19.347359 23581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:19.349170 23581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:34001
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:19.349551 23581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:19.349859 23581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:19.356117 23598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:19.356134 23590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:19.356110 23591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:19.356374 23581 server_base.cc:1034] running on GCE node
I20260808 00:42:19.356794 23581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:19.357192 23581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:19.358438 23581 hybrid_clock.cc:648] HybridClock initialized: now 1786149739358416 us; error 57 us; skew 500 ppm
Aug 08 00:42:19 dist-test-slave-kvfs krb5kdc[23556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149739, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:19.361979 23581 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:19.362716 23581 webserver.cc:460] Webserver started at http://127.20.105.126:40395/ using document root <none> and password file <none>
I20260808 00:42:19.363044 23581 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:19.363131 23581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:19.363306 23581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:19.364706 23581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "1fd6732a00bf49e988ff84ac55cb669c"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "2ebc77788bc9a2b8f1c3c6b342587169"
server_key_iv: "db32e990dd9037dc224e674425bdc46a"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.365175 23581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "1fd6732a00bf49e988ff84ac55cb669c"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "2ebc77788bc9a2b8f1c3c6b342587169"
server_key_iv: "db32e990dd9037dc224e674425bdc46a"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.366832 23581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:19.367879 23609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:19.368178 23581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:19.368302 23581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "1fd6732a00bf49e988ff84ac55cb669c"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "2ebc77788bc9a2b8f1c3c6b342587169"
server_key_iv: "db32e990dd9037dc224e674425bdc46a"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.368397 23581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:19.384518 23581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:19.387624 23581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:19.387844 23581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:19.394198 23581 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:34001
I20260808 00:42:19.394397 23698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:34001 every 8 connection(s)
I20260808 00:42:19.394806 23581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:19.395861 23700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:19.399152 23700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: Bootstrap starting.
I20260808 00:42:19.399941 23700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:19.400322 23700 log.cc:824] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:19.400835 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 23581
I20260808 00:42:19.400969 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:19.401214 20901 external_mini_cluster.cc:1428] Setting key 04965d52a1e38892dbe9ec9968725b43
I20260808 00:42:19.401502 23700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: No bootstrap required, opened a new log
I20260808 00:42:19.404618 23700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd6732a00bf49e988ff84ac55cb669c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34001 } }
I20260808 00:42:19.404838 23700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:19.404910 23700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd6732a00bf49e988ff84ac55cb669c, State: Initialized, Role: FOLLOWER
I20260808 00:42:19.405110 23700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd6732a00bf49e988ff84ac55cb669c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34001 } }
I20260808 00:42:19.405221 23700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:19.405274 23700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:19.405354 23700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:19.406153 23700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd6732a00bf49e988ff84ac55cb669c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34001 } }
I20260808 00:42:19.406320 23700 leader_election.cc:302] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fd6732a00bf49e988ff84ac55cb669c; no voters:
I20260808 00:42:19.406564 23700 leader_election.cc:288] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:19.406600 23709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:19.406735 23709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [term 1 LEADER]: Becoming Leader. State: Replica: 1fd6732a00bf49e988ff84ac55cb669c, State: Running, Role: LEADER
I20260808 00:42:19.406903 23709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd6732a00bf49e988ff84ac55cb669c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34001 } }
I20260808 00:42:19.406952 23700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:19.407423 23710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fd6732a00bf49e988ff84ac55cb669c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd6732a00bf49e988ff84ac55cb669c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34001 } } }
I20260808 00:42:19.407505 23710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:19.407704 23713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd6732a00bf49e988ff84ac55cb669c. Latest consensus state: current_term: 1 leader_uuid: "1fd6732a00bf49e988ff84ac55cb669c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd6732a00bf49e988ff84ac55cb669c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34001 } } }
I20260808 00:42:19.407783 23713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:19.408419 23717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:19.409173 23717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:42:19 dist-test-slave-kvfs krb5kdc[23556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149739, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:19.413285 23717 catalog_manager.cc:1347] Generated new cluster ID: e79aa20c55494483acbed552e8f1c133
I20260808 00:42:19.413355 23717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:19.417762 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.401939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49708 (local address 127.20.105.126:34001)
0808 00:42:19.402172 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:19.402184 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:19.402225 (+ 41us) server_negotiation.cc:407] Connection header received
0808 00:42:19.404584 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:19.404597 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:19.404693 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:19.404960 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:19.405887 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.407460 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.408797 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.409695 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.409883 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.413314 (+ 3431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:19.413318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:19.413319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:19.413335 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:19.415682 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.416203 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.416207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.416207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.416365 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.416630 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.416631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.416631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.417022 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:19.417193 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:19.417343 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 00:42:19.417509 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":137,"threads_started":1}
I20260808 00:42:19.436147 23717 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:19.437028 23717 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:19.449396 23717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: Generated new TSK 0
I20260808 00:42:19.449751 23717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:19.488660 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34001
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:19.591876 23749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:19.592196 23749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:19.592229 23749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:19.593657 23749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:19.593732 23749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:19.593758 23749 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:42:19.593771 23749 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:19.593791 23749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:19.595729 23749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:34001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:19.596156 23749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:19.596553 23749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:19.603370 23761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:19.603394 23757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:19.603564 23749 server_base.cc:1034] running on GCE node
W20260808 00:42:19.603395 23756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:19.603912 23749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:19.604298 23749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:19.605494 23749 hybrid_clock.cc:648] HybridClock initialized: now 1786149739605468 us; error 35 us; skew 500 ppm
Aug 08 00:42:19 dist-test-slave-kvfs krb5kdc[23556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149739, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:19.612795 23749 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:19.613499 23749 webserver.cc:460] Webserver started at http://127.20.105.65:38831/ using document root <none> and password file <none>
I20260808 00:42:19.613839 23749 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:19.613916 23749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:19.614084 23749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:19.615392 23749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "9162c300df504b87bfe465b5592b0f85"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "a68f26ebfdb18c6f6d9663a53d4169b2"
server_key_iv: "a67a20587eb9cb76331ad85ccc046bf9"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.615831 23749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "9162c300df504b87bfe465b5592b0f85"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "a68f26ebfdb18c6f6d9663a53d4169b2"
server_key_iv: "a67a20587eb9cb76331ad85ccc046bf9"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.617679 23749 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:19.618712 23769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:19.618952 23749 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:19.619081 23749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "9162c300df504b87bfe465b5592b0f85"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "a68f26ebfdb18c6f6d9663a53d4169b2"
server_key_iv: "a67a20587eb9cb76331ad85ccc046bf9"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.619176 23749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:19.644682 23749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:19.647629 23749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:19.647854 23749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:19.648712 23749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:19.648790 23749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:19.648835 23749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:19.648866 23749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:42:19 dist-test-slave-kvfs krb5kdc[23556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149739, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:19.658396 23749 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:44295
I20260808 00:42:19.658430 23957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:44295 every 8 connection(s)
I20260808 00:42:19.658902 23749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:19.662160 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.649329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:41567 (local address 127.20.105.126:34001)
0808 00:42:19.649421 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:19.649424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:19.649764 (+ 340us) server_negotiation.cc:407] Connection header received
0808 00:42:19.650796 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:19.650798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:19.650863 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:19.650945 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:19.652436 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.653203 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.655125 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.655360 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.655466 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.658585 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:19.658587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:19.658589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:19.658604 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:19.660339 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.660830 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.660832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.660832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.660934 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.661292 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.661293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.661294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.661451 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:19.661582 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:19.661938 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 00:42:19.662077 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:42:19.662338 23785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.649287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34001 (local address 127.20.105.65:41567)
0808 00:42:19.649552 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:19.649628 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:19.650616 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:19.651111 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:19.651118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:19.651679 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:19.652301 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:19.652313 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.653295 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.653296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:19.655000 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:19.655005 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.655493 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.655495 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:19.655655 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.656387 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:19.656399 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:19.658430 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:19.660449 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:19.660458 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:19.660460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:19.660733 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:19.661030 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:19.661032 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:19.661032 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:19.661170 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:19.661524 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:19.661530 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:19.661759 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 00:42:19.661911 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 00:42:19.662080 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":126,"threads_started":1}
I20260808 00:42:19.663045 23959 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34001
I20260808 00:42:19.663187 23959 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:19.663422 23959 heartbeater.cc:511] Master 127.20.105.126:34001 requested a full tablet report, sending...
I20260808 00:42:19.664244 23633 ts_manager.cc:194] Registered new tserver with Master: 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295)
I20260808 00:42:19.665210 23633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:41567
I20260808 00:42:19.667582 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 23749
I20260808 00:42:19.667719 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:19.667950 20901 external_mini_cluster.cc:1428] Setting key 8ca50cc1d79ba64547bc498f176b4398
I20260808 00:42:19.671598 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.664410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:45415 (local address 127.20.105.126:34001)
0808 00:42:19.664668 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:19.664670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:19.664678 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:19.664699 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:19.664700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:19.664711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:19.664768 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:19.665240 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.665842 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.666609 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.666833 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.666924 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.668273 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:19.668274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:19.668275 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:19.668283 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:19.670093 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.670516 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.670518 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.670518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.670593 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.670853 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.670854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.670854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.671021 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:19.671093 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:19.671322 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 00:42:19.671441 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
I20260808 00:42:19.671597 23969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.664335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34001 (local address 127.20.105.65:45415)
0808 00:42:19.664494 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:19.664506 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:19.664580 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:19.664831 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:19.664833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:19.665059 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:19.665136 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:19.665138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.665951 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.665953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:19.666516 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:19.666518 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.666926 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.666927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:19.667005 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.667564 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:19.667580 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:19.668121 (+ 541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:19.670215 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:19.670216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:19.670216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:19.670428 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:19.670694 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:19.670695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:19.670695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:19.670775 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:19.671127 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:19.671128 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:19.671189 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 00:42:19.671267 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 00:42:19.671412 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:19.705192 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34001
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:19.832094 23972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:19.832443 23972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:19.832475 23972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:19.833828 23972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:19.833902 23972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:19.833927 23972 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:42:19.833940 23972 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:19.833958 23972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:19.835832 23972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:34001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:19.836289 23972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:19.836643 23972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:19.843290 23979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:19.843284 23982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:19.843351 23972 server_base.cc:1034] running on GCE node
W20260808 00:42:19.843312 23980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:19.844002 23972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:19.844360 23972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:19.845556 23972 hybrid_clock.cc:648] HybridClock initialized: now 1786149739845538 us; error 48 us; skew 500 ppm
Aug 08 00:42:19 dist-test-slave-kvfs krb5kdc[23556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149739, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:19.848784 23972 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:19.849308 23972 webserver.cc:460] Webserver started at http://127.20.105.66:39865/ using document root <none> and password file <none>
I20260808 00:42:19.849560 23972 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:19.849606 23972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:19.849736 23972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:19.850962 23972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "4e683c4033704d8a8dbf066ac37851df"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "3e13450253b33dbdd6557cbde247db4c"
server_key_iv: "58d170fa6f1ca0abd800630a2eb51721"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.851342 23972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "4e683c4033704d8a8dbf066ac37851df"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "3e13450253b33dbdd6557cbde247db4c"
server_key_iv: "58d170fa6f1ca0abd800630a2eb51721"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.853137 23972 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 00:42:19.854017 23997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:19.854245 23972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:19.854333 23972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "4e683c4033704d8a8dbf066ac37851df"
format_stamp: "Formatted at 2026-08-08 00:42:19 on dist-test-slave-kvfs"
server_key: "3e13450253b33dbdd6557cbde247db4c"
server_key_iv: "58d170fa6f1ca0abd800630a2eb51721"
server_key_version: "encryptionkey@0"
I20260808 00:42:19.854399 23972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:19.878146 23972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:19.881167 23972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:19.881314 23972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:19.882030 23972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:19.882079 23972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:19.882108 23972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:19.882120 23972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:19.890893 23972 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:35617
Aug 08 00:42:19 dist-test-slave-kvfs krb5kdc[23556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149739, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:19.890996 24166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:35617 every 8 connection(s)
I20260808 00:42:19.891409 23972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:19.894722 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.882665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:49995 (local address 127.20.105.126:34001)
0808 00:42:19.882761 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:19.882765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:19.883058 (+ 293us) server_negotiation.cc:407] Connection header received
0808 00:42:19.883813 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:19.883815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:19.883837 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:19.883917 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:19.885404 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.886377 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.888221 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.888575 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.888674 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.891710 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:19.891713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:19.891713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:19.891723 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:19.893443 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.893856 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.893857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.893857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.893917 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.894122 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.894122 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.894123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.894249 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:19.894304 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:19.894565 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 00:42:19.894643 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:42:19.894943 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.882623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34001 (local address 127.20.105.66:49995)
0808 00:42:19.882847 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:19.882900 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:19.883641 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:19.884201 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:19.884223 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:19.884870 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:19.885259 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:19.885268 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.886512 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.886513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:19.888128 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:19.888133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.888680 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.888681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:19.888814 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.889566 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:19.889579 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:19.891563 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:19.893527 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:19.893533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:19.893535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:19.893783 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:19.893970 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:19.893971 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:19.893971 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:19.894068 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:19.894291 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:19.894294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:19.894426 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 00:42:19.894529 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 00:42:19.894672 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":133,"threads_started":1}
I20260808 00:42:19.895668 24167 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34001
I20260808 00:42:19.895771 24167 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:19.896003 24167 heartbeater.cc:511] Master 127.20.105.126:34001 requested a full tablet report, sending...
I20260808 00:42:19.896580 23634 ts_manager.cc:194] Registered new tserver with Master: 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:19.897181 23634 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:49995
I20260808 00:42:19.897466 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 23972
I20260808 00:42:19.897593 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:19.897807 20901 external_mini_cluster.cc:1428] Setting key 14396f2879991797fc7f5697c86df166
I20260808 00:42:19.903855 24177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.896852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34001 (local address 127.20.105.66:43833)
0808 00:42:19.897011 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:19.897023 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:19.897092 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:19.897216 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:19.897217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:19.897442 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:19.897509 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:19.897511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.898387 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.898388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:19.899235 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:19.899239 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.899616 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.899617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:19.899724 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.900270 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:19.900276 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:19.900741 (+ 465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:19.902537 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:19.902538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:19.902539 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:19.902777 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:19.903071 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:19.903073 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:19.903073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:19.903150 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:19.903572 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:19.903573 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:19.903617 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 00:42:19.903676 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:42:19.903773 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260808 00:42:19.903954 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:19.896940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:43833 (local address 127.20.105.126:34001)
0808 00:42:19.896990 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:19.896997 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:19.897077 (+ 80us) server_negotiation.cc:407] Connection header received
0808 00:42:19.897149 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:19.897150 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:19.897158 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:19.897200 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:19.897593 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.898287 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.899340 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:19.899540 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:19.899595 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:19.900846 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:19.900848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:19.900848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:19.900855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:19.902437 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.902885 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.902886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.902887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.902960 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:19.903227 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:19.903229 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:19.903230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:19.903501 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:19.903576 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:19.903730 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 00:42:19.903823 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:19.935382 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34001
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:20.036410 24181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:20.036736 24181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:20.036773 24181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:20.038334 24181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:20.038395 24181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:20.038422 24181 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:42:20.038439 24181 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:20.038458 24181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:20.040490 24181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:34001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:20.040853 24181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:20.041180 24181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:20.047540 24191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:20.047670 24181 server_base.cc:1034] running on GCE node
W20260808 00:42:20.047552 24189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:20.047788 24188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:20.048112 24181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:20.048444 24181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:20.049625 24181 hybrid_clock.cc:648] HybridClock initialized: now 1786149740049601 us; error 33 us; skew 500 ppm
Aug 08 00:42:20 dist-test-slave-kvfs krb5kdc[23556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149740, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:20.052883 24181 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:20.053596 24181 webserver.cc:460] Webserver started at http://127.20.105.67:44427/ using document root <none> and password file <none>
I20260808 00:42:20.053885 24181 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:20.053968 24181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:20.054137 24181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:20.055476 24181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "46711fbbf7bf4f8aa734999d2330a422"
format_stamp: "Formatted at 2026-08-08 00:42:20 on dist-test-slave-kvfs"
server_key: "ebc043d0325a0b8f6ae597c72c300d50"
server_key_iv: "0be566224d650855f0118ce730ef86a9"
server_key_version: "encryptionkey@0"
I20260808 00:42:20.055933 24181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "46711fbbf7bf4f8aa734999d2330a422"
format_stamp: "Formatted at 2026-08-08 00:42:20 on dist-test-slave-kvfs"
server_key: "ebc043d0325a0b8f6ae597c72c300d50"
server_key_iv: "0be566224d650855f0118ce730ef86a9"
server_key_version: "encryptionkey@0"
I20260808 00:42:20.057682 24181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:20.058672 24204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:20.058967 24181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:20.059098 24181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "46711fbbf7bf4f8aa734999d2330a422"
format_stamp: "Formatted at 2026-08-08 00:42:20 on dist-test-slave-kvfs"
server_key: "ebc043d0325a0b8f6ae597c72c300d50"
server_key_iv: "0be566224d650855f0118ce730ef86a9"
server_key_version: "encryptionkey@0"
I20260808 00:42:20.059201 24181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:20.073449 24181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:20.076699 24181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:20.076938 24181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:20.077852 24181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:20.077930 24181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:20.077972 24181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:20.077989 24181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:42:20 dist-test-slave-kvfs krb5kdc[23556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149740, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:20.087483 24181 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:44173
I20260808 00:42:20.087819 24384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:44173 every 8 connection(s)
I20260808 00:42:20.088092 24181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:20.090097 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.078334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:50513 (local address 127.20.105.126:34001)
0808 00:42:20.078443 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.078446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.078740 (+ 294us) server_negotiation.cc:407] Connection header received
0808 00:42:20.079492 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.079493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.079514 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.079613 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:20.080766 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.081404 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.082639 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.082908 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.083004 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.086421 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:20.086424 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:20.086424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:20.086440 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:20.088566 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:20.089013 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:20.089014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:20.089015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:20.089083 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:20.089346 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:20.089348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:20.089348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:20.089504 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:20.089594 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.089925 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.090017 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 00:42:20.090314 24217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.078348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34001 (local address 127.20.105.67:50513)
0808 00:42:20.078546 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.078595 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.079330 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.079736 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.079741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.080177 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:20.080634 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.080642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.081515 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.081516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.082525 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.082527 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.083014 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.083015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.083207 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.083947 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:20.083961 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:20.086297 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:20.088675 (+ 2378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:20.088680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:20.088683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:20.088937 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:20.089154 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:20.089155 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:20.089155 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:20.089261 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:20.089604 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:20.089611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:20.089762 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.089871 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.090059 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":91,"threads_started":1}
I20260808 00:42:20.090983 24386 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34001
I20260808 00:42:20.091081 24386 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:20.091303 24386 heartbeater.cc:511] Master 127.20.105.126:34001 requested a full tablet report, sending...
I20260808 00:42:20.091760 23633 ts_manager.cc:194] Registered new tserver with Master: 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173)
I20260808 00:42:20.092311 23633 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:50513
I20260808 00:42:20.094290 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 24181
I20260808 00:42:20.094422 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:20.094627 20901 external_mini_cluster.cc:1428] Setting key c1ea69fa187021a540cfbded061a277a
I20260808 00:42:20.095836 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:42:20.098726 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.092286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:32833 (local address 127.20.105.126:34001)
0808 00:42:20.092343 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.092347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.092461 (+ 114us) server_negotiation.cc:407] Connection header received
0808 00:42:20.092542 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.092543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.092554 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.092602 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:20.092986 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.093502 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.094257 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.094535 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.094627 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.096102 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:20.096106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:20.096106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:20.096117 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:20.097502 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:20.097862 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:20.097864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:20.097864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:20.097930 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:20.098152 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:20.098153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:20.098153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:20.098277 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:20.098355 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.098527 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.098619 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:42:20.098738 24395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.092215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34001 (local address 127.20.105.67:32833)
0808 00:42:20.092375 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.092386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.092478 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.092606 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.092606 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.092812 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:20.092880 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.092882 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.093609 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.093610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.094156 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.094157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.094627 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.094628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.094713 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.095292 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:20.095298 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:20.095736 (+ 438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:20.097595 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:20.097596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:20.097597 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:20.097782 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:20.098001 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:20.098002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:20.098002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:20.098068 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:20.098355 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:20.098357 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:20.098410 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.098499 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.098629 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":88,"threads_started":1}
Aug 08 00:42:20 dist-test-slave-kvfs krb5kdc[23556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:42:20 dist-test-slave-kvfs krb5kdc[23556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149740, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:20.128785 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.119587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.20.105.126:34001)
0808 00:42:20.119700 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.119702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.119735 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:42:20.119773 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.119774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.119786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.119835 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:20.120625 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.121201 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.121968 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.122185 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.122251 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.125527 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:20.125530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:20.125530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:20.125546 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:20.127370 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:20.127878 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:20.127879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:20.127880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:20.127955 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:20.128234 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:20.128234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:20.128235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:20.128384 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:20.128487 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.128611 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.128696 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 00:42:20.131129 23633 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:20.133486 23633 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:20.142045 24416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.137736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.20.105.65:44295)
0808 00:42:20.137947 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.137953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.138465 (+ 512us) server_negotiation.cc:407] Connection header received
0808 00:42:20.138540 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.138545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.138563 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.138668 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:20.138882 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.139651 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.140927 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.141512 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.141604 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.141685 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.141844 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.141915 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":74,"threads_started":1}
I20260808 00:42:20.142035 24414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.137558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:44295 (local address 127.0.0.1:38350)
0808 00:42:20.138113 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.138167 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.138286 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.138758 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.138759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.138761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:20.138801 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.138802 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.139748 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.139749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.140829 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.140832 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.141606 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.141608 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.141694 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.141710 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.141819 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.141919 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":160,"threads_started":1}
I20260808 00:42:20.142910 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.137576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44173 (local address 127.0.0.1:41640)
0808 00:42:20.138254 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.138267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.138380 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.138640 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.138644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.138647 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:20.138768 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.138771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.140332 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.140333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.141601 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.141604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.142641 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.142642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.142711 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.142716 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.142773 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.142828 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":58,"thread_start_us":143,"threads_started":1}
I20260808 00:42:20.142983 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.137726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.20.105.67:44173)
0808 00:42:20.137997 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.138003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.138404 (+ 401us) server_negotiation.cc:407] Connection header received
0808 00:42:20.138485 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.138488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.138510 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.138616 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:20.138873 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.140222 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.141721 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.142530 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.142617 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.142682 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.142787 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.142849 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":123,"threads_started":1}
I20260808 00:42:20.143163 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.138259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.20.105.66:35617)
0808 00:42:20.138527 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.138532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.138547 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:42:20.138595 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.138597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.138613 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.138698 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:20.138972 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.140230 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.141720 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.142577 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.142643 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.142691 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.142964 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.143020 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":102,"threads_started":1}
I20260808 00:42:20.143389 24415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.137902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:35617 (local address 127.0.0.1:47698)
0808 00:42:20.138406 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.138417 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.138467 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.138784 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.138785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.138788 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:20.138869 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.138872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.140342 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.140343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.141602 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.141604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.142733 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.142734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.142803 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.142808 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.142863 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.142913 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":46,"thread_start_us":134,"threads_started":1}
I20260808 00:42:20.143399 23851 tablet_service.cc:1467] Processing CreateTablet for tablet 29c4914e0a154235b2c4930034d45efd (DEFAULT_TABLE table=test-table [id=caaac2909ed84118ac3b5b07c2ca4ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:20.144006 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29c4914e0a154235b2c4930034d45efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.143895 24268 tablet_service.cc:1467] Processing CreateTablet for tablet 29c4914e0a154235b2c4930034d45efd (DEFAULT_TABLE table=test-table [id=caaac2909ed84118ac3b5b07c2ca4ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:20.144205 24072 tablet_service.cc:1467] Processing CreateTablet for tablet 29c4914e0a154235b2c4930034d45efd (DEFAULT_TABLE table=test-table [id=caaac2909ed84118ac3b5b07c2ca4ce0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:20.144378 24268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29c4914e0a154235b2c4930034d45efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.144567 24072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 29c4914e0a154235b2c4930034d45efd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.147537 24424 tablet_bootstrap.cc:492] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Bootstrap starting.
I20260808 00:42:20.148841 24424 tablet_bootstrap.cc:654] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.149225 24424 log.cc:824] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:20.149257 24425 tablet_bootstrap.cc:492] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Bootstrap starting.
I20260808 00:42:20.150249 24424 tablet_bootstrap.cc:492] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: No bootstrap required, opened a new log
I20260808 00:42:20.150348 24424 ts_tablet_manager.cc:1397] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:42:20.150378 24425 tablet_bootstrap.cc:654] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.150707 24425 log.cc:824] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:20.151249 24426 tablet_bootstrap.cc:492] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Bootstrap starting.
I20260808 00:42:20.151888 24426 tablet_bootstrap.cc:654] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.152297 24426 log.cc:824] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:20.152333 24424 raft_consensus.cc:348] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.152518 24425 tablet_bootstrap.cc:492] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: No bootstrap required, opened a new log
I20260808 00:42:20.152541 24424 raft_consensus.cc:374] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.152588 24425 ts_tablet_manager.cc:1397] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:42:20.152585 24424 raft_consensus.cc:729] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9162c300df504b87bfe465b5592b0f85, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.152735 24424 consensus_queue.cc:260] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.153074 24424 ts_tablet_manager.cc:1428] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 00:42:20.153330 23959 heartbeater.cc:503] Master 127.20.105.126:34001 was elected leader, sending a full tablet report...
I20260808 00:42:20.153364 24426 tablet_bootstrap.cc:492] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: No bootstrap required, opened a new log
I20260808 00:42:20.153434 24426 ts_tablet_manager.cc:1397] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:20.154799 24425 raft_consensus.cc:348] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.155009 24425 raft_consensus.cc:374] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.155064 24425 raft_consensus.cc:729] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46711fbbf7bf4f8aa734999d2330a422, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.155169 24426 raft_consensus.cc:348] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.155274 24425 consensus_queue.cc:260] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.155346 24426 raft_consensus.cc:374] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.155377 24426 raft_consensus.cc:729] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e683c4033704d8a8dbf066ac37851df, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.155521 24426 consensus_queue.cc:260] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.155687 24425 ts_tablet_manager.cc:1428] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:42:20.155699 24386 heartbeater.cc:503] Master 127.20.105.126:34001 was elected leader, sending a full tablet report...
I20260808 00:42:20.155975 24426 ts_tablet_manager.cc:1428] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:42:20.156080 24167 heartbeater.cc:503] Master 127.20.105.126:34001 was elected leader, sending a full tablet report...
W20260808 00:42:20.159811 23960 tablet.cc:2369] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:42:20.338910 24387 tablet.cc:2369] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:20.380928 24435 raft_consensus.cc:482] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:20.381146 24435 raft_consensus.cc:504] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.381629 24435 leader_election.cc:288] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173), 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:20.385075 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.382093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:46991 (local address 127.20.105.67:44173)
0808 00:42:20.382199 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.382202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.382215 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:42:20.382279 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.382280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.382290 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.382373 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:20.382539 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.383086 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.384193 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.384598 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.384690 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.384702 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.384917 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.384974 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 00:42:20.385324 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.382059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:44451 (local address 127.20.105.66:35617)
0808 00:42:20.382131 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.382134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.382292 (+ 158us) server_negotiation.cc:407] Connection header received
0808 00:42:20.382392 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.382393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.382405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.382484 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:20.382688 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.383228 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.384340 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.384785 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.384903 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.384914 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.385181 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.385241 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:42:20.385366 24442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.381915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44173 (local address 127.20.105.65:46991)
0808 00:42:20.382096 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.382112 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.382203 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.382364 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.382366 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.382368 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:20.382460 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.382462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.383186 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.383188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.384086 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.384089 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.384689 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.384690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.384802 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.384808 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.385208 (+ 400us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.385254 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260808 00:42:20.385749 24440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.381919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:35617 (local address 127.20.105.65:44451)
0808 00:42:20.382182 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.382200 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.382313 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.382507 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.382508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.382510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:20.382611 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.382613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.383326 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.383328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.384229 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.384232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.384970 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.384971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.385079 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.385085 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.385153 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.385225 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":92,"threads_started":1}
I20260808 00:42:20.385795 24305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29c4914e0a154235b2c4930034d45efd" candidate_uuid: "9162c300df504b87bfe465b5592b0f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46711fbbf7bf4f8aa734999d2330a422" is_pre_election: true
I20260808 00:42:20.386051 24305 raft_consensus.cc:2454] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9162c300df504b87bfe465b5592b0f85 in term 0.
I20260808 00:42:20.386217 24097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29c4914e0a154235b2c4930034d45efd" candidate_uuid: "9162c300df504b87bfe465b5592b0f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df" is_pre_election: true
I20260808 00:42:20.386422 24097 raft_consensus.cc:2454] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9162c300df504b87bfe465b5592b0f85 in term 0.
I20260808 00:42:20.386437 23775 leader_election.cc:302] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46711fbbf7bf4f8aa734999d2330a422, 9162c300df504b87bfe465b5592b0f85; no voters:
I20260808 00:42:20.386603 24435 raft_consensus.cc:2781] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:20.386695 24435 raft_consensus.cc:482] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:20.386736 24435 raft_consensus.cc:3037] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.387701 24435 raft_consensus.cc:504] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.387892 24435 leader_election.cc:288] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [CANDIDATE]: Term 1 election: Requested vote from peers 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173), 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:20.388130 24305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29c4914e0a154235b2c4930034d45efd" candidate_uuid: "9162c300df504b87bfe465b5592b0f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46711fbbf7bf4f8aa734999d2330a422"
I20260808 00:42:20.388123 24097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "29c4914e0a154235b2c4930034d45efd" candidate_uuid: "9162c300df504b87bfe465b5592b0f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df"
I20260808 00:42:20.388222 24097 raft_consensus.cc:3037] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.388222 24305 raft_consensus.cc:3037] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.389312 24305 raft_consensus.cc:2454] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9162c300df504b87bfe465b5592b0f85 in term 1.
I20260808 00:42:20.389312 24097 raft_consensus.cc:2454] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9162c300df504b87bfe465b5592b0f85 in term 1.
I20260808 00:42:20.389529 23775 leader_election.cc:302] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46711fbbf7bf4f8aa734999d2330a422, 9162c300df504b87bfe465b5592b0f85; no voters:
I20260808 00:42:20.389703 24435 raft_consensus.cc:2781] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:20.389881 24435 raft_consensus.cc:686] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 1 LEADER]: Becoming Leader. State: Replica: 9162c300df504b87bfe465b5592b0f85, State: Running, Role: LEADER
I20260808 00:42:20.390010 24435 consensus_queue.cc:237] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } }
I20260808 00:42:20.391235 23632 catalog_manager.cc:5495] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9162c300df504b87bfe465b5592b0f85 (127.20.105.65). New cstate: current_term: 1 leader_uuid: "9162c300df504b87bfe465b5592b0f85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } health_report { overall_health: UNKNOWN } } }
W20260808 00:42:20.392270 24168 tablet.cc:2369] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 00:42:20 dist-test-slave-kvfs krb5kdc[23556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149739, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:20.458071 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.449246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34001 (local address 127.0.0.1:49730)
0808 00:42:20.449521 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.449544 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.449656 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.449842 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.449843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.450308 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:20.450397 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.450402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.451153 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.451154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.451755 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.451758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.452220 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.452222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.452299 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.452933 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:20.452939 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:20.454727 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:20.456742 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:20.456752 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:20.456755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:20.456985 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:20.457217 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:20.457218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:20.457219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:20.457286 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:20.457595 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:20.457605 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:20.457668 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.457740 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.457924 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":5,"thread_start_us":143,"threads_started":1}
I20260808 00:42:20.458071 23708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.449359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.20.105.126:34001)
0808 00:42:20.449492 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.449495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.449629 (+ 134us) server_negotiation.cc:407] Connection header received
0808 00:42:20.449740 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.449741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.449753 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.449835 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:20.450500 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.451059 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.451856 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.452128 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.452219 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.454869 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:20.454872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:20.454872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:20.454882 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:20.456653 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:20.457082 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:20.457083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:20.457084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:20.457148 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:20.457353 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:20.457354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:20.457354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:20.457495 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:20.457575 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.457779 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.457982 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":85}
I20260808 00:42:20.460847 23632 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:49730:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:42:20.461197 23632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:20.463222 23851 tablet_service.cc:1467] Processing CreateTablet for tablet 067d2c42dae54679aa842d631a6fe760 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=604ae363051341058e27247a17b576bf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:20.463215 24072 tablet_service.cc:1467] Processing CreateTablet for tablet 067d2c42dae54679aa842d631a6fe760 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=604ae363051341058e27247a17b576bf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:20.463209 24268 tablet_service.cc:1467] Processing CreateTablet for tablet 067d2c42dae54679aa842d631a6fe760 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=604ae363051341058e27247a17b576bf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:20.463454 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067d2c42dae54679aa842d631a6fe760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.463454 24072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067d2c42dae54679aa842d631a6fe760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.463466 24268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 067d2c42dae54679aa842d631a6fe760. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.465258 24425 tablet_bootstrap.cc:492] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: Bootstrap starting.
I20260808 00:42:20.465369 24424 tablet_bootstrap.cc:492] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: Bootstrap starting.
I20260808 00:42:20.465801 24424 tablet_bootstrap.cc:654] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.465866 24425 tablet_bootstrap.cc:654] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.466207 24426 tablet_bootstrap.cc:492] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: Bootstrap starting.
I20260808 00:42:20.466572 24425 tablet_bootstrap.cc:492] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: No bootstrap required, opened a new log
I20260808 00:42:20.466646 24425 ts_tablet_manager.cc:1397] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:20.466642 24426 tablet_bootstrap.cc:654] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.466832 24425 raft_consensus.cc:348] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.466939 24425 raft_consensus.cc:374] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.466969 24425 raft_consensus.cc:729] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46711fbbf7bf4f8aa734999d2330a422, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.467020 24425 consensus_queue.cc:260] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.467152 24425 ts_tablet_manager.cc:1428] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:42:20.467284 24437 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.467383 24426 tablet_bootstrap.cc:492] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: No bootstrap required, opened a new log
I20260808 00:42:20.467404 24437 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:20.467434 24426 ts_tablet_manager.cc:1397] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:20.467628 24426 raft_consensus.cc:348] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.467696 24426 raft_consensus.cc:374] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.467710 24426 raft_consensus.cc:729] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e683c4033704d8a8dbf066ac37851df, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.467785 24426 consensus_queue.cc:260] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.467808 24424 tablet_bootstrap.cc:492] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: No bootstrap required, opened a new log
I20260808 00:42:20.467861 24424 ts_tablet_manager.cc:1397] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260808 00:42:20.467902 24426 ts_tablet_manager.cc:1428] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:20.467963 24439 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.468076 24439 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:20.468243 24424 raft_consensus.cc:348] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.468339 24424 raft_consensus.cc:374] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.468374 24424 raft_consensus.cc:729] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9162c300df504b87bfe465b5592b0f85, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.468434 24424 consensus_queue.cc:260] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.468564 24424 ts_tablet_manager.cc:1428] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:20.468641 24435 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.468750 24435 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:20.787380 24437 raft_consensus.cc:482] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:20.787598 24437 raft_consensus.cc:504] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.788153 24437 leader_election.cc:288] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295), 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:20.791864 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.788619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:37891 (local address 127.20.105.65:44295)
0808 00:42:20.788766 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.788769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.788786 (+ 17us) server_negotiation.cc:407] Connection header received
0808 00:42:20.788830 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.788831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.788845 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.788993 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:20.789223 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.789782 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.790962 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.791445 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.791555 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.791567 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.791716 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.791756 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260808 00:42:20.791894 24460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.788480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:44295 (local address 127.20.105.67:37891)
0808 00:42:20.788645 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.788661 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.788744 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.789005 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.789007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.789011 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:20.789137 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.789139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.789873 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.789875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.790859 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.790862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.791552 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.791553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.791636 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.791640 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.791709 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.791756 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":20,"thread_start_us":102,"threads_started":1}
I20260808 00:42:20.791913 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.788732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:35617 (local address 127.20.105.67:58603)
0808 00:42:20.788861 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.788901 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.788964 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.789136 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.789137 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.789140 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:20.789232 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.789233 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.789961 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.789962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.790892 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.790894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.791584 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.791585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.791669 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.791672 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.791741 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.791787 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":3,"thread_start_us":53,"threads_started":1}
I20260808 00:42:20.791898 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.788857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:58603 (local address 127.20.105.66:35617)
0808 00:42:20.788956 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.788959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.788971 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:20.789040 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.789041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.789057 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.789131 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:20.789311 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.789856 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.790991 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.791494 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.791584 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.791595 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.791764 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.791813 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 00:42:20.792275 24097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "067d2c42dae54679aa842d631a6fe760" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df" is_pre_election: true
I20260808 00:42:20.792420 24097 raft_consensus.cc:2454] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 0.
I20260808 00:42:20.792572 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "067d2c42dae54679aa842d631a6fe760" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9162c300df504b87bfe465b5592b0f85" is_pre_election: true
I20260808 00:42:20.792666 24205 leader_election.cc:302] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46711fbbf7bf4f8aa734999d2330a422, 4e683c4033704d8a8dbf066ac37851df; no voters:
I20260808 00:42:20.792708 23880 raft_consensus.cc:2454] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 0.
I20260808 00:42:20.792829 24437 raft_consensus.cc:2781] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:20.792876 24437 raft_consensus.cc:482] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:20.792920 24437 raft_consensus.cc:3037] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.793818 24437 raft_consensus.cc:504] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.793970 24437 leader_election.cc:288] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 election: Requested vote from peers 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295), 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:20.794152 24097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "067d2c42dae54679aa842d631a6fe760" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df"
I20260808 00:42:20.794231 24097 raft_consensus.cc:3037] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.794178 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "067d2c42dae54679aa842d631a6fe760" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9162c300df504b87bfe465b5592b0f85"
I20260808 00:42:20.794287 23880 raft_consensus.cc:3037] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.794950 24097 raft_consensus.cc:2454] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 1.
I20260808 00:42:20.795110 24205 leader_election.cc:302] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46711fbbf7bf4f8aa734999d2330a422, 4e683c4033704d8a8dbf066ac37851df; no voters:
I20260808 00:42:20.795205 24437 raft_consensus.cc:2781] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:20.795336 24437 raft_consensus.cc:686] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [term 1 LEADER]: Becoming Leader. State: Replica: 46711fbbf7bf4f8aa734999d2330a422, State: Running, Role: LEADER
I20260808 00:42:20.795377 23880 raft_consensus.cc:2454] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 1.
I20260808 00:42:20.795476 24437 consensus_queue.cc:237] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.796061 24464 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: TxnStatusTablet state changed. Reason: New leader 46711fbbf7bf4f8aa734999d2330a422. Latest consensus state: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.796154 24464 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:20.796464 24466 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:42:20.796643 24466 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 00:42:20.796736 23634 catalog_manager.cc:5495] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } health_report { overall_health: HEALTHY } } }
I20260808 00:42:20.831108 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.827073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44173 (local address 127.0.0.1:41656)
0808 00:42:20.827190 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.827210 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.827279 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.827609 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.827610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.827612 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:20.827688 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.827691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.828862 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.828864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.829617 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.829620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.830280 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.830282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.830357 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.830396 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.830846 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.830862 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.830950 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.831024 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 00:42:20.831115 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.827138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41656 (local address 127.20.105.67:44173)
0808 00:42:20.827206 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.827209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.827380 (+ 171us) server_negotiation.cc:407] Connection header received
0808 00:42:20.827449 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.827450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.827483 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.827600 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:20.827767 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.828716 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.829732 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.830136 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.830232 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.830541 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.830733 (+ 192us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.830853 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.830956 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.830997 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:42:20.834404 23880 raft_consensus.cc:1261] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Refusing update from remote peer 46711fbbf7bf4f8aa734999d2330a422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:20.834404 24097 raft_consensus.cc:1261] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Refusing update from remote peer 46711fbbf7bf4f8aa734999d2330a422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:20.834865 24437 consensus_queue.cc:1035] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:20.835049 24464 consensus_queue.cc:1035] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:20.836607 24439 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: TxnStatusTablet state changed. Reason: New leader 46711fbbf7bf4f8aa734999d2330a422. Latest consensus state: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.836740 24439 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:20.836874 24435 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: TxnStatusTablet state changed. Reason: New leader 46711fbbf7bf4f8aa734999d2330a422. Latest consensus state: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.837034 24435 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:20.838325 24472 mvcc.cc:205] Tried to move back new op lower bound from 7316069338453422080 to 7316069338298941440. Current Snapshot: MvccSnapshot[applied={T|T < 7316069338453422080}]
I20260808 00:42:20.838423 24437 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.838546 24437 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:20.839039 24464 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.839133 24464 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 46711fbbf7bf4f8aa734999d2330a422: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:20.839279 24473 mvcc.cc:205] Tried to move back new op lower bound from 7316069338453422080 to 7316069338298941440. Current Snapshot: MvccSnapshot[applied={T|T < 7316069338453422080}]
I20260808 00:42:20.839385 24435 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.839476 24435 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 9162c300df504b87bfe465b5592b0f85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:20.840407 24439 tablet_replica.cc:439] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } } }
I20260808 00:42:20.840459 24474 mvcc.cc:205] Tried to move back new op lower bound from 7316069338453422080 to 7316069338298941440. Current Snapshot: MvccSnapshot[applied={T|T < 7316069338453422080}]
I20260808 00:42:20.840520 24439 tablet_replica.cc:442] T 067d2c42dae54679aa842d631a6fe760 P 4e683c4033704d8a8dbf066ac37851df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:20.840695 24435 consensus_queue.cc:1035] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:20.842935 24443 consensus_queue.cc:1035] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:20.847759 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.844465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38360 (local address 127.20.105.65:44295)
0808 00:42:20.844508 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.844510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.844831 (+ 321us) server_negotiation.cc:407] Connection header received
0808 00:42:20.844920 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.844921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.844943 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.845023 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:20.845165 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.845969 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.846807 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.847130 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.847206 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.847357 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.847515 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.847622 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.847642 (+ 20us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.847669 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:42:20.854038 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.851003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:42205 (local address 127.20.105.67:44173)
0808 00:42:20.851097 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.851099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.851237 (+ 138us) server_negotiation.cc:407] Connection header received
0808 00:42:20.851272 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.851273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.851286 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.851344 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:20.851493 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.852008 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.853273 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.853475 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.853534 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.853781 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.853815 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.853860 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.853942 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.853964 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 00:42:20.854092 24491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.850920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44173 (local address 127.20.105.65:42205)
0808 00:42:20.851136 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.851148 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.851218 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.851347 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.851348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.851350 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:20.851430 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.851432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.852149 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.852151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.853174 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.853177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.853562 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.853563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.853641 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.853662 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.853880 (+ 218us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.853883 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.853937 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.853986 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":117,"threads_started":1}
I20260808 00:42:20.866667 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.863423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:60597 (local address 127.20.105.65:44295)
0808 00:42:20.863473 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.863475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.863600 (+ 125us) server_negotiation.cc:407] Connection header received
0808 00:42:20.863674 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.863675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.863688 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.863740 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:20.863916 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.864517 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.865741 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.866031 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.866098 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.866319 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.866381 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.866434 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.866520 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.866567 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:42:20.866736 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.863273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:44295 (local address 127.20.105.67:60597)
0808 00:42:20.863478 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.863522 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.863619 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.863741 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.863741 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.863743 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:20.863842 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.863844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.864611 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.864612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.865650 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.865653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.866138 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.866139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.866215 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.866238 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.866466 (+ 228us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.866469 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.866542 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.866591 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":102,"threads_started":1}
I20260808 00:42:20.879616 23632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:20.879805 23632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:20.880955 23632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: Sending DeleteTablet for 3 replicas of tablet 29c4914e0a154235b2c4930034d45efd
I20260808 00:42:20.881366 23851 tablet_service.cc:1514] Processing DeleteTablet for tablet 29c4914e0a154235b2c4930034d45efd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:20 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:38350
I20260808 00:42:20.881851 24502 tablet_replica.cc:330] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: stopping tablet replica
I20260808 00:42:20.881888 24072 tablet_service.cc:1514] Processing DeleteTablet for tablet 29c4914e0a154235b2c4930034d45efd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:20 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:47698
I20260808 00:42:20.882134 24502 raft_consensus.cc:2229] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:20.882324 24502 raft_consensus.cc:2258] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:20.882772 23632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:20.882983 23632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:20.883196 24502 ts_tablet_manager.cc:1905] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:20.883913 24268 tablet_service.cc:1514] Processing DeleteTablet for tablet 29c4914e0a154235b2c4930034d45efd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:20 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:41640
I20260808 00:42:20.885399 23851 tablet_service.cc:1467] Processing CreateTablet for tablet 185cc6e844d044d4844340f4389516f3 (DEFAULT_TABLE table=test-table [id=1220e5a560354f8ca0c0f8305734be9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:20.885571 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185cc6e844d044d4844340f4389516f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.886301 24502 ts_tablet_manager.cc:1918] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:20.886473 24502 log.cc:1196] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/29c4914e0a154235b2c4930034d45efd
I20260808 00:42:20.886868 24502 ts_tablet_manager.cc:1939] T 29c4914e0a154235b2c4930034d45efd P 9162c300df504b87bfe465b5592b0f85: Deleting consensus metadata
I20260808 00:42:20.887176 24424 tablet_bootstrap.cc:492] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: Bootstrap starting.
I20260808 00:42:20.887552 23610 catalog_manager.cc:4841] TS 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295): tablet 29c4914e0a154235b2c4930034d45efd (table test-table [id=caaac2909ed84118ac3b5b07c2ca4ce0]) successfully deleted
I20260808 00:42:20.887718 24424 tablet_bootstrap.cc:654] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.887782 24072 tablet_service.cc:1467] Processing CreateTablet for tablet 185cc6e844d044d4844340f4389516f3 (DEFAULT_TABLE table=test-table [id=1220e5a560354f8ca0c0f8305734be9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:20.887957 24072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185cc6e844d044d4844340f4389516f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.888446 24424 tablet_bootstrap.cc:492] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: No bootstrap required, opened a new log
I20260808 00:42:20.888497 24424 ts_tablet_manager.cc:1397] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:20.888665 24424 raft_consensus.cc:348] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.888744 24424 raft_consensus.cc:374] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.888765 24424 raft_consensus.cc:729] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9162c300df504b87bfe465b5592b0f85, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.888814 24424 consensus_queue.cc:260] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.888913 24424 ts_tablet_manager.cc:1428] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:20.889741 24426 tablet_bootstrap.cc:492] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: Bootstrap starting.
I20260808 00:42:20.890239 24426 tablet_bootstrap.cc:654] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.891122 24268 tablet_service.cc:1467] Processing CreateTablet for tablet 185cc6e844d044d4844340f4389516f3 (DEFAULT_TABLE table=test-table [id=1220e5a560354f8ca0c0f8305734be9d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:20.891279 24268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185cc6e844d044d4844340f4389516f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:20.892130 24506 tablet_replica.cc:330] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: stopping tablet replica
I20260808 00:42:20.892297 24506 raft_consensus.cc:2229] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:20.892385 24506 raft_consensus.cc:2258] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:20.893018 24425 tablet_bootstrap.cc:492] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: Bootstrap starting.
I20260808 00:42:20.893151 24506 ts_tablet_manager.cc:1905] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:20.893486 24425 tablet_bootstrap.cc:654] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:20.893880 24426 tablet_bootstrap.cc:492] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: No bootstrap required, opened a new log
I20260808 00:42:20.893953 24426 ts_tablet_manager.cc:1397] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260808 00:42:20.894114 24426 raft_consensus.cc:348] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.894219 24426 raft_consensus.cc:374] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.894245 24426 raft_consensus.cc:729] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e683c4033704d8a8dbf066ac37851df, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.894299 24426 consensus_queue.cc:260] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.894344 24425 tablet_bootstrap.cc:492] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: No bootstrap required, opened a new log
I20260808 00:42:20.894394 24425 ts_tablet_manager.cc:1397] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:20.894411 24426 ts_tablet_manager.cc:1428] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:20.894779 24425 raft_consensus.cc:348] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.894877 24425 raft_consensus.cc:374] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:20.894902 24425 raft_consensus.cc:729] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46711fbbf7bf4f8aa734999d2330a422, State: Initialized, Role: FOLLOWER
I20260808 00:42:20.894963 24425 consensus_queue.cc:260] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.895076 24425 ts_tablet_manager.cc:1428] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:20.895157 24506 ts_tablet_manager.cc:1918] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:20.895231 24506 log.cc:1196] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/29c4914e0a154235b2c4930034d45efd
I20260808 00:42:20.895557 24506 ts_tablet_manager.cc:1939] T 29c4914e0a154235b2c4930034d45efd P 46711fbbf7bf4f8aa734999d2330a422: Deleting consensus metadata
I20260808 00:42:20.895591 24503 tablet_replica.cc:330] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: stopping tablet replica
I20260808 00:42:20.895700 24503 raft_consensus.cc:2229] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:20.895764 24503 raft_consensus.cc:2258] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:20.896255 24503 ts_tablet_manager.cc:1905] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:20.896283 23616 catalog_manager.cc:4841] TS 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173): tablet 29c4914e0a154235b2c4930034d45efd (table test-table [id=caaac2909ed84118ac3b5b07c2ca4ce0]) successfully deleted
I20260808 00:42:20.898202 24503 ts_tablet_manager.cc:1918] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:20.898283 24503 log.cc:1196] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/29c4914e0a154235b2c4930034d45efd
I20260808 00:42:20.898568 24503 ts_tablet_manager.cc:1939] T 29c4914e0a154235b2c4930034d45efd P 4e683c4033704d8a8dbf066ac37851df: Deleting consensus metadata
I20260808 00:42:20.899080 23610 catalog_manager.cc:4841] TS 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617): tablet 29c4914e0a154235b2c4930034d45efd (table test-table [id=caaac2909ed84118ac3b5b07c2ca4ce0]) successfully deleted
I20260808 00:42:20.923089 24464 raft_consensus.cc:482] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:20.923240 24464 raft_consensus.cc:504] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.923470 24464 leader_election.cc:288] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295), 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:20.923725 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185cc6e844d044d4844340f4389516f3" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9162c300df504b87bfe465b5592b0f85" is_pre_election: true
I20260808 00:42:20.923779 24097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185cc6e844d044d4844340f4389516f3" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df" is_pre_election: true
I20260808 00:42:20.923895 24097 raft_consensus.cc:2454] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 0.
I20260808 00:42:20.923893 23880 raft_consensus.cc:2454] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 0.
I20260808 00:42:20.924114 24205 leader_election.cc:302] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46711fbbf7bf4f8aa734999d2330a422, 9162c300df504b87bfe465b5592b0f85; no voters:
I20260808 00:42:20.924331 24464 raft_consensus.cc:2781] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:20.924379 24464 raft_consensus.cc:482] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:20.924402 24464 raft_consensus.cc:3037] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.925668 24464 raft_consensus.cc:504] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.925845 24464 leader_election.cc:288] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 election: Requested vote from peers 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295), 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:20.926153 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185cc6e844d044d4844340f4389516f3" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9162c300df504b87bfe465b5592b0f85"
I20260808 00:42:20.926158 24097 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185cc6e844d044d4844340f4389516f3" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df"
I20260808 00:42:20.926266 24097 raft_consensus.cc:3037] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.926252 23880 raft_consensus.cc:3037] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:20.927343 23880 raft_consensus.cc:2454] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 1.
I20260808 00:42:20.927347 24097 raft_consensus.cc:2454] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 1.
I20260808 00:42:20.927604 24205 leader_election.cc:302] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46711fbbf7bf4f8aa734999d2330a422, 9162c300df504b87bfe465b5592b0f85; no voters:
I20260808 00:42:20.927815 24464 raft_consensus.cc:2781] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:20.927884 24464 raft_consensus.cc:686] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 1 LEADER]: Becoming Leader. State: Replica: 46711fbbf7bf4f8aa734999d2330a422, State: Running, Role: LEADER
I20260808 00:42:20.927948 24464 consensus_queue.cc:237] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:20.928917 23632 catalog_manager.cc:5495] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } health_report { overall_health: HEALTHY } } }
I20260808 00:42:20.939285 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.935738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.20.105.67:44173)
0808 00:42:20.935786 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.935788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.935823 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:42:20.935901 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.935902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.935915 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.935989 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:20.936275 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.937121 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.938388 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.938708 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.938785 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.938976 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.939027 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.939082 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.939154 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.939181 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:42:20.945782 24417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.942742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:49121 (local address 127.20.105.67:44173)
0808 00:42:20.942779 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:20.942781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:20.942789 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:20.942844 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:20.942845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:20.942856 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:20.942909 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:20.943048 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.943785 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.944888 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.945194 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.945270 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.945449 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.945487 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.945541 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:20.945615 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 00:42:20.945637 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:42:20.945896 24496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:20.942643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44173 (local address 127.20.105.67:49121)
0808 00:42:20.942712 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:20.942728 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:20.942787 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:20.942910 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:20.942911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:20.942912 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:20.942967 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.942970 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.943883 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.943884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.944784 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:20.944788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:20.945288 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:20.945289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:20.945349 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:20.945358 (+ 9us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:20.945560 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:20.945562 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:42:20.945611 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:42:20.945681 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:42:20.949540 24097 raft_consensus.cc:1261] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Refusing update from remote peer 46711fbbf7bf4f8aa734999d2330a422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:20.949919 23878 raft_consensus.cc:1261] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Refusing update from remote peer 46711fbbf7bf4f8aa734999d2330a422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:20.949879 24492 consensus_queue.cc:1035] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:20.950511 24492 consensus_queue.cc:1035] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:42:20.958791 24472 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 00:42:22.481714 23632 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:22.481860 23632 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:22.482925 23632 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: Sending DeleteTablet for 3 replicas of tablet 185cc6e844d044d4844340f4389516f3
I20260808 00:42:22.483273 24268 tablet_service.cc:1514] Processing DeleteTablet for tablet 185cc6e844d044d4844340f4389516f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:22 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:41640
I20260808 00:42:22.483266 23851 tablet_service.cc:1514] Processing DeleteTablet for tablet 185cc6e844d044d4844340f4389516f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:22 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:38350
I20260808 00:42:22.483312 24072 tablet_service.cc:1514] Processing DeleteTablet for tablet 185cc6e844d044d4844340f4389516f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:22 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:47698
I20260808 00:42:22.483532 24549 tablet_replica.cc:330] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: stopping tablet replica
I20260808 00:42:22.483601 24549 raft_consensus.cc:2229] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:22.483664 24549 raft_consensus.cc:2258] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:22.483773 24548 tablet_replica.cc:330] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: stopping tablet replica
I20260808 00:42:22.483837 24548 raft_consensus.cc:2229] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:22.483942 24548 raft_consensus.cc:2258] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:22.484242 24549 ts_tablet_manager.cc:1905] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:22.485867 24548 ts_tablet_manager.cc:1905] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:22.486125 24549 ts_tablet_manager.cc:1918] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:42:22.486192 24549 log.cc:1196] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/185cc6e844d044d4844340f4389516f3
I20260808 00:42:22.486543 24549 ts_tablet_manager.cc:1939] T 185cc6e844d044d4844340f4389516f3 P 4e683c4033704d8a8dbf066ac37851df: Deleting consensus metadata
I20260808 00:42:22.486937 23610 catalog_manager.cc:4841] TS 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617): tablet 185cc6e844d044d4844340f4389516f3 (table test-table [id=1220e5a560354f8ca0c0f8305734be9d]) successfully deleted
I20260808 00:42:22.487805 24548 ts_tablet_manager.cc:1918] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:42:22.487860 24548 log.cc:1196] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/185cc6e844d044d4844340f4389516f3
I20260808 00:42:22.488221 24548 ts_tablet_manager.cc:1939] T 185cc6e844d044d4844340f4389516f3 P 46711fbbf7bf4f8aa734999d2330a422: Deleting consensus metadata
I20260808 00:42:22.488613 23616 catalog_manager.cc:4841] TS 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173): tablet 185cc6e844d044d4844340f4389516f3 (table test-table [id=1220e5a560354f8ca0c0f8305734be9d]) successfully deleted
I20260808 00:42:22.490998 24550 tablet_replica.cc:330] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: stopping tablet replica
I20260808 00:42:22.491096 24550 raft_consensus.cc:2229] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:22.491169 24550 raft_consensus.cc:2258] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:22.491487 24550 ts_tablet_manager.cc:1905] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:22.493444 24550 ts_tablet_manager.cc:1918] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:42:22.493546 24550 log.cc:1196] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/185cc6e844d044d4844340f4389516f3
I20260808 00:42:22.493804 24550 ts_tablet_manager.cc:1939] T 185cc6e844d044d4844340f4389516f3 P 9162c300df504b87bfe465b5592b0f85: Deleting consensus metadata
I20260808 00:42:22.494222 23610 catalog_manager.cc:4841] TS 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295): tablet 185cc6e844d044d4844340f4389516f3 (table test-table [id=1220e5a560354f8ca0c0f8305734be9d]) successfully deleted
Aug 08 00:42:22 dist-test-slave-kvfs krb5kdc[23556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:42:22 dist-test-slave-kvfs krb5kdc[23556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149742, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:22.515957 24561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:22.506850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.20.105.126:34001)
0808 00:42:22.507171 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:22.507173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:22.507187 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:42:22.507267 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:22.507268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:22.507283 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:22.507384 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:22.508291 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:22.508896 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:22.509750 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:22.509989 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:22.510080 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:22.512865 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:22.512867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:22.512867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:22.512877 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:22.514570 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:22.514982 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:22.514982 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:22.514983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:22.515052 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:22.515333 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:22.515334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:22.515334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:22.515481 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:22.515576 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:22.515741 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 00:42:22.515851 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":201,"threads_started":1}
I20260808 00:42:22.518170 23632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:22.518402 23632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:22.520504 24072 tablet_service.cc:1467] Processing CreateTablet for tablet 7dfbbc0db4d74361872078a3b864a00a (DEFAULT_TABLE table=test-table [id=44c8be20afde45779ab4d44d5a636c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:22.520484 24268 tablet_service.cc:1467] Processing CreateTablet for tablet 7dfbbc0db4d74361872078a3b864a00a (DEFAULT_TABLE table=test-table [id=44c8be20afde45779ab4d44d5a636c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:22.520714 24072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dfbbc0db4d74361872078a3b864a00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:22.520803 24268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dfbbc0db4d74361872078a3b864a00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:22.522204 24562 tablet_bootstrap.cc:492] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: Bootstrap starting.
I20260808 00:42:22.522702 24563 tablet_bootstrap.cc:492] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: Bootstrap starting.
I20260808 00:42:22.522748 23851 tablet_service.cc:1467] Processing CreateTablet for tablet 7dfbbc0db4d74361872078a3b864a00a (DEFAULT_TABLE table=test-table [id=44c8be20afde45779ab4d44d5a636c4f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:22.522797 24562 tablet_bootstrap.cc:654] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:22.522934 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7dfbbc0db4d74361872078a3b864a00a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:22.523211 24563 tablet_bootstrap.cc:654] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:22.524138 24563 tablet_bootstrap.cc:492] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: No bootstrap required, opened a new log
I20260808 00:42:22.524209 24563 ts_tablet_manager.cc:1397] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:42:22.524415 24563 raft_consensus.cc:348] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.524536 24563 raft_consensus.cc:374] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:22.524564 24563 raft_consensus.cc:729] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46711fbbf7bf4f8aa734999d2330a422, State: Initialized, Role: FOLLOWER
I20260808 00:42:22.524614 24563 consensus_queue.cc:260] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.524735 24563 ts_tablet_manager.cc:1428] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:22.525005 24571 tablet_bootstrap.cc:492] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: Bootstrap starting.
I20260808 00:42:22.525581 24571 tablet_bootstrap.cc:654] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:22.525676 24562 tablet_bootstrap.cc:492] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: No bootstrap required, opened a new log
I20260808 00:42:22.525755 24562 ts_tablet_manager.cc:1397] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 00:42:22.526086 24562 raft_consensus.cc:348] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.526196 24562 raft_consensus.cc:374] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:22.526221 24562 raft_consensus.cc:729] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e683c4033704d8a8dbf066ac37851df, State: Initialized, Role: FOLLOWER
I20260808 00:42:22.526314 24562 consensus_queue.cc:260] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.526408 24571 tablet_bootstrap.cc:492] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: No bootstrap required, opened a new log
I20260808 00:42:22.526477 24571 ts_tablet_manager.cc:1397] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:22.526549 24562 ts_tablet_manager.cc:1428] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:22.526661 24571 raft_consensus.cc:348] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.526763 24571 raft_consensus.cc:374] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:22.526796 24571 raft_consensus.cc:729] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9162c300df504b87bfe465b5592b0f85, State: Initialized, Role: FOLLOWER
I20260808 00:42:22.526854 24571 consensus_queue.cc:260] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.527153 24571 ts_tablet_manager.cc:1428] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:22.606762 24492 raft_consensus.cc:482] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:22.606956 24492 raft_consensus.cc:504] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.607205 24492 leader_election.cc:288] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295), 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:22.607611 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dfbbc0db4d74361872078a3b864a00a" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df" is_pre_election: true
I20260808 00:42:22.607601 23878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dfbbc0db4d74361872078a3b864a00a" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9162c300df504b87bfe465b5592b0f85" is_pre_election: true
I20260808 00:42:22.607731 24095 raft_consensus.cc:2454] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 0.
I20260808 00:42:22.607755 23878 raft_consensus.cc:2454] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 0.
I20260808 00:42:22.607954 24205 leader_election.cc:302] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46711fbbf7bf4f8aa734999d2330a422, 4e683c4033704d8a8dbf066ac37851df; no voters:
I20260808 00:42:22.608126 24492 raft_consensus.cc:2781] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:22.608174 24492 raft_consensus.cc:482] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:22.608201 24492 raft_consensus.cc:3037] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:22.609233 24492 raft_consensus.cc:504] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.609383 24492 leader_election.cc:288] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 election: Requested vote from peers 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295), 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617)
I20260808 00:42:22.609602 23878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dfbbc0db4d74361872078a3b864a00a" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9162c300df504b87bfe465b5592b0f85"
I20260808 00:42:22.609582 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7dfbbc0db4d74361872078a3b864a00a" candidate_uuid: "46711fbbf7bf4f8aa734999d2330a422" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df"
I20260808 00:42:22.609696 24095 raft_consensus.cc:3037] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:22.609697 23878 raft_consensus.cc:3037] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:22.610720 24095 raft_consensus.cc:2454] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 1.
I20260808 00:42:22.610743 23878 raft_consensus.cc:2454] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46711fbbf7bf4f8aa734999d2330a422 in term 1.
I20260808 00:42:22.610929 24205 leader_election.cc:302] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46711fbbf7bf4f8aa734999d2330a422, 9162c300df504b87bfe465b5592b0f85; no voters:
I20260808 00:42:22.611040 24492 raft_consensus.cc:2781] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:22.611099 24492 raft_consensus.cc:686] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 1 LEADER]: Becoming Leader. State: Replica: 46711fbbf7bf4f8aa734999d2330a422, State: Running, Role: LEADER
I20260808 00:42:22.611164 24492 consensus_queue.cc:237] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.612125 23632 catalog_manager.cc:5495] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 reported cstate change: term changed from 0 to 1, leader changed from <none> to 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "46711fbbf7bf4f8aa734999d2330a422" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } health_report { overall_health: HEALTHY } } }
I20260808 00:42:22.622277 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:22.619469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47968 (local address 127.20.105.67:44173)
0808 00:42:22.619729 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:22.619731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:22.619737 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:42:22.619758 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:22.619759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:22.619771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:22.619852 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:22.620050 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:22.620631 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:22.621466 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:22.621700 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:22.621784 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:22.621943 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:22.621989 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:22.622044 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:22.622136 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 00:42:22.622166 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":189,"threads_started":1}
I20260808 00:42:22.625034 24095 raft_consensus.cc:1261] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Refusing update from remote peer 46711fbbf7bf4f8aa734999d2330a422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:22.625099 23878 raft_consensus.cc:1261] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Refusing update from remote peer 46711fbbf7bf4f8aa734999d2330a422: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:22.625309 24529 consensus_queue.cc:1035] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:22.625466 24524 consensus_queue.cc:1035] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:22.638233 23634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:22.638417 23634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:22.639358 23634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: Sending DeleteTablet for 3 replicas of tablet 7dfbbc0db4d74361872078a3b864a00a
I20260808 00:42:22.639864 24268 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dfbbc0db4d74361872078a3b864a00a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:22 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:41640
I20260808 00:42:22.640094 24548 tablet_replica.cc:330] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: stopping tablet replica
I20260808 00:42:22.640117 24072 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dfbbc0db4d74361872078a3b864a00a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:22 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:47698
I20260808 00:42:22.640170 24548 raft_consensus.cc:2229] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:22.640223 24549 tablet_replica.cc:330] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: stopping tablet replica
I20260808 00:42:22.640257 24548 raft_consensus.cc:2258] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:22.640154 23634 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 00:42:22.640285 24549 raft_consensus.cc:2229] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:22.640338 24549 raft_consensus.cc:2258] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 00:42:22.640383 23634 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:22.639971 23851 tablet_service.cc:1514] Processing DeleteTablet for tablet 7dfbbc0db4d74361872078a3b864a00a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:22 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:38350
I20260808 00:42:22.640573 24550 tablet_replica.cc:330] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: stopping tablet replica
I20260808 00:42:22.640656 24550 raft_consensus.cc:2229] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:22.640707 24550 raft_consensus.cc:2258] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:22.640740 24549 ts_tablet_manager.cc:1905] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:22.641067 24550 ts_tablet_manager.cc:1905] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:22.641189 24548 ts_tablet_manager.cc:1905] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:22.642032 24072 tablet_service.cc:1467] Processing CreateTablet for tablet ab687b31a2754749b2688ed1489e161e (DEFAULT_TABLE table=test-table [id=4d2ce9dff24048eea1f8a1670516ee79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:22.642196 24072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab687b31a2754749b2688ed1489e161e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:22.642979 24549 ts_tablet_manager.cc:1918] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:22.643039 24549 log.cc:1196] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/7dfbbc0db4d74361872078a3b864a00a
I20260808 00:42:22.643177 24548 ts_tablet_manager.cc:1918] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:22.643195 24550 ts_tablet_manager.cc:1918] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:22.643231 24548 log.cc:1196] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/7dfbbc0db4d74361872078a3b864a00a
I20260808 00:42:22.643251 24550 log.cc:1196] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/7dfbbc0db4d74361872078a3b864a00a
I20260808 00:42:22.643304 24549 ts_tablet_manager.cc:1939] T 7dfbbc0db4d74361872078a3b864a00a P 4e683c4033704d8a8dbf066ac37851df: Deleting consensus metadata
I20260808 00:42:22.643494 24550 ts_tablet_manager.cc:1939] T 7dfbbc0db4d74361872078a3b864a00a P 9162c300df504b87bfe465b5592b0f85: Deleting consensus metadata
I20260808 00:42:22.643510 24548 ts_tablet_manager.cc:1939] T 7dfbbc0db4d74361872078a3b864a00a P 46711fbbf7bf4f8aa734999d2330a422: Deleting consensus metadata
I20260808 00:42:22.643775 24562 tablet_bootstrap.cc:492] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: Bootstrap starting.
I20260808 00:42:22.644055 23610 catalog_manager.cc:4841] TS 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295): tablet 7dfbbc0db4d74361872078a3b864a00a (table test-table [id=44c8be20afde45779ab4d44d5a636c4f]) successfully deleted
I20260808 00:42:22.644076 23851 tablet_service.cc:1467] Processing CreateTablet for tablet ab687b31a2754749b2688ed1489e161e (DEFAULT_TABLE table=test-table [id=4d2ce9dff24048eea1f8a1670516ee79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:22.644177 23851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab687b31a2754749b2688ed1489e161e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:22.644532 24562 tablet_bootstrap.cc:654] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:22.645282 24571 tablet_bootstrap.cc:492] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85: Bootstrap starting.
I20260808 00:42:22.645612 24562 tablet_bootstrap.cc:492] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: No bootstrap required, opened a new log
I20260808 00:42:22.645658 24562 ts_tablet_manager.cc:1397] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:22.645743 24571 tablet_bootstrap.cc:654] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:22.645859 24562 raft_consensus.cc:348] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.645941 24562 raft_consensus.cc:374] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:22.645962 24562 raft_consensus.cc:729] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e683c4033704d8a8dbf066ac37851df, State: Initialized, Role: FOLLOWER
I20260808 00:42:22.646025 24562 consensus_queue.cc:260] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.646126 24562 ts_tablet_manager.cc:1428] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:22.646227 23610 catalog_manager.cc:4841] TS 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617): tablet 7dfbbc0db4d74361872078a3b864a00a (table test-table [id=44c8be20afde45779ab4d44d5a636c4f]) successfully deleted
I20260808 00:42:22.646273 23616 catalog_manager.cc:4841] TS 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173): tablet 7dfbbc0db4d74361872078a3b864a00a (table test-table [id=44c8be20afde45779ab4d44d5a636c4f]) successfully deleted
I20260808 00:42:22.646461 24268 tablet_service.cc:1467] Processing CreateTablet for tablet ab687b31a2754749b2688ed1489e161e (DEFAULT_TABLE table=test-table [id=4d2ce9dff24048eea1f8a1670516ee79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:22.646597 24268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab687b31a2754749b2688ed1489e161e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:22.646721 24571 tablet_bootstrap.cc:492] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85: No bootstrap required, opened a new log
I20260808 00:42:22.646762 24571 ts_tablet_manager.cc:1397] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:22.646879 24571 raft_consensus.cc:348] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.646940 24571 raft_consensus.cc:374] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:22.646958 24571 raft_consensus.cc:729] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9162c300df504b87bfe465b5592b0f85, State: Initialized, Role: FOLLOWER
I20260808 00:42:22.647010 24571 consensus_queue.cc:260] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.647104 24571 ts_tablet_manager.cc:1428] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:22.648015 24563 tablet_bootstrap.cc:492] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: Bootstrap starting.
I20260808 00:42:22.648483 24563 tablet_bootstrap.cc:654] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:22.649163 24563 tablet_bootstrap.cc:492] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: No bootstrap required, opened a new log
I20260808 00:42:22.649235 24563 ts_tablet_manager.cc:1397] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:22.649379 24563 raft_consensus.cc:348] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.649459 24563 raft_consensus.cc:374] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:22.649483 24563 raft_consensus.cc:729] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46711fbbf7bf4f8aa734999d2330a422, State: Initialized, Role: FOLLOWER
I20260808 00:42:22.649539 24563 consensus_queue.cc:260] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.649645 24563 ts_tablet_manager.cc:1428] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:42:22.704521 24577 raft_consensus.cc:482] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:22.704681 24577 raft_consensus.cc:504] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.704962 24577 leader_election.cc:288] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617), 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173)
I20260808 00:42:22.705286 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab687b31a2754749b2688ed1489e161e" candidate_uuid: "9162c300df504b87bfe465b5592b0f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df" is_pre_election: true
I20260808 00:42:22.705271 24305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab687b31a2754749b2688ed1489e161e" candidate_uuid: "9162c300df504b87bfe465b5592b0f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46711fbbf7bf4f8aa734999d2330a422" is_pre_election: true
I20260808 00:42:22.705423 24095 raft_consensus.cc:2454] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9162c300df504b87bfe465b5592b0f85 in term 0.
I20260808 00:42:22.705428 24305 raft_consensus.cc:2454] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9162c300df504b87bfe465b5592b0f85 in term 0.
I20260808 00:42:22.705660 23771 leader_election.cc:302] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e683c4033704d8a8dbf066ac37851df, 9162c300df504b87bfe465b5592b0f85; no voters:
I20260808 00:42:22.705793 24577 raft_consensus.cc:2781] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:22.705848 24577 raft_consensus.cc:482] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:22.705864 24577 raft_consensus.cc:3037] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:22.706972 24577 raft_consensus.cc:504] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.707211 24577 leader_election.cc:288] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [CANDIDATE]: Term 1 election: Requested vote from peers 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617), 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173)
I20260808 00:42:22.707404 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab687b31a2754749b2688ed1489e161e" candidate_uuid: "9162c300df504b87bfe465b5592b0f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e683c4033704d8a8dbf066ac37851df"
I20260808 00:42:22.707511 24095 raft_consensus.cc:3037] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:22.707494 24305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab687b31a2754749b2688ed1489e161e" candidate_uuid: "9162c300df504b87bfe465b5592b0f85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46711fbbf7bf4f8aa734999d2330a422"
I20260808 00:42:22.707612 24305 raft_consensus.cc:3037] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:22.708761 24095 raft_consensus.cc:2454] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9162c300df504b87bfe465b5592b0f85 in term 1.
I20260808 00:42:22.708834 24305 raft_consensus.cc:2454] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9162c300df504b87bfe465b5592b0f85 in term 1.
I20260808 00:42:22.708987 23771 leader_election.cc:302] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e683c4033704d8a8dbf066ac37851df, 9162c300df504b87bfe465b5592b0f85; no voters:
I20260808 00:42:22.709110 24577 raft_consensus.cc:2781] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:22.709259 24577 raft_consensus.cc:686] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [term 1 LEADER]: Becoming Leader. State: Replica: 9162c300df504b87bfe465b5592b0f85, State: Running, Role: LEADER
I20260808 00:42:22.709311 24577 consensus_queue.cc:237] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } }
I20260808 00:42:22.710129 23632 catalog_manager.cc:5495] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9162c300df504b87bfe465b5592b0f85 (127.20.105.65). New cstate: current_term: 1 leader_uuid: "9162c300df504b87bfe465b5592b0f85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9162c300df504b87bfe465b5592b0f85" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 44295 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:22.728871 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:22.726322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32886 (local address 127.20.105.65:44295)
0808 00:42:22.726593 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:22.726595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:22.726602 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:42:22.726624 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:22.726625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:22.726636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:22.726695 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:22.726847 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:22.727376 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:22.728180 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:22.728398 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:22.728457 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:22.728566 (+ 109us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:22.728602 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:22.728637 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:22.728711 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 00:42:22.728729 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":208,"threads_started":1}
I20260808 00:42:22.731822 24305 raft_consensus.cc:1261] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Refusing update from remote peer 9162c300df504b87bfe465b5592b0f85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:22.731828 24095 raft_consensus.cc:1261] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Refusing update from remote peer 9162c300df504b87bfe465b5592b0f85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:22.732089 24593 consensus_queue.cc:1035] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e683c4033704d8a8dbf066ac37851df" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:22.732245 24577 consensus_queue.cc:1035] T ab687b31a2754749b2688ed1489e161e P 9162c300df504b87bfe465b5592b0f85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46711fbbf7bf4f8aa734999d2330a422" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:42:22.737290 24526 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 00:42:24.257813 23634 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:24.257956 23634 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:24.259385 23634 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1fd6732a00bf49e988ff84ac55cb669c: Sending DeleteTablet for 3 replicas of tablet ab687b31a2754749b2688ed1489e161e
I20260808 00:42:24.259749 24268 tablet_service.cc:1514] Processing DeleteTablet for tablet ab687b31a2754749b2688ed1489e161e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:24 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:41640
I20260808 00:42:24.259804 24072 tablet_service.cc:1514] Processing DeleteTablet for tablet ab687b31a2754749b2688ed1489e161e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:24 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:47698
I20260808 00:42:24.259974 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 23749
I20260808 00:42:24.272305 24637 tablet_replica.cc:330] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: stopping tablet replica
I20260808 00:42:24.272466 24637 raft_consensus.cc:2229] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:24.272543 24637 raft_consensus.cc:2258] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:24.272746 24636 tablet_replica.cc:330] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: stopping tablet replica
I20260808 00:42:24.272840 24636 raft_consensus.cc:2229] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:24.272903 24636 raft_consensus.cc:2258] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:24.273236 24637 ts_tablet_manager.cc:1905] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:24.273255 24636 ts_tablet_manager.cc:1905] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:24.275295 24637 ts_tablet_manager.cc:1918] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:42:24.275434 24637 log.cc:1196] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/ab687b31a2754749b2688ed1489e161e
I20260808 00:42:24.275739 24636 ts_tablet_manager.cc:1918] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:42:24.275791 24637 ts_tablet_manager.cc:1939] T ab687b31a2754749b2688ed1489e161e P 4e683c4033704d8a8dbf066ac37851df: Deleting consensus metadata
I20260808 00:42:24.275837 24636 log.cc:1196] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/ab687b31a2754749b2688ed1489e161e
I20260808 00:42:24.276157 24636 ts_tablet_manager.cc:1939] T ab687b31a2754749b2688ed1489e161e P 46711fbbf7bf4f8aa734999d2330a422: Deleting consensus metadata
I20260808 00:42:24.276533 23610 catalog_manager.cc:4841] TS 4e683c4033704d8a8dbf066ac37851df (127.20.105.66:35617): tablet ab687b31a2754749b2688ed1489e161e (table test-table [id=4d2ce9dff24048eea1f8a1670516ee79]) successfully deleted
I20260808 00:42:24.276888 23616 catalog_manager.cc:4841] TS 46711fbbf7bf4f8aa734999d2330a422 (127.20.105.67:44173): tablet ab687b31a2754749b2688ed1489e161e (table test-table [id=4d2ce9dff24048eea1f8a1670516ee79]) successfully deleted
W20260808 00:42:24.281246 23610 connection.cc:535] client connection to 127.20.105.65:44295 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 00:42:24.281414 23610 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:42:24.281700 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 23972
I20260808 00:42:24.293188 24641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:24.281891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:44295 (local address 127.0.0.1:32898)
0808 00:42:24.292908 (+ 11017us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:24.293085 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:44295: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10947,"thread_start_us":90,"threads_started":1}
W20260808 00:42:24.293452 23610 catalog_manager.cc:4584] TS 9162c300df504b87bfe465b5592b0f85 (127.20.105.65:44295): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ab687b31a2754749b2688ed1489e161e: Network error: Client connection negotiation failed: client connection to 127.20.105.65:44295: connect: Connection refused (error 111)
I20260808 00:42:24.304199 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 24181
I20260808 00:42:24.330075 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 23581
2026-08-08T00:42:24Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6551 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:24 dist-test-slave-kvfs krb5kdc[24646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:24 dist-test-slave-kvfs krb5kdc[24646](info): set up 2 sockets
Aug 08 00:42:24 dist-test-slave-kvfs krb5kdc[24646](info): commencing operation
krb5kdc: starting...
W20260808 00:42:25.659211 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:25 dist-test-slave-kvfs krb5kdc[24646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:25.811894 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:36461
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:36461
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:25.915805 24668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:25.916222 24668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:25.916275 24668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:25.918717 24668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:25.919020 24668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:25.919076 24668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:25.919109 24668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:25.919142 24668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:25.921418 24668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:36461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:36461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:25.921840 24668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:25.922166 24668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:25.928685 24674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:25.928851 24668 server_base.cc:1034] running on GCE node
W20260808 00:42:25.928699 24677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:25.928689 24675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:25.929430 24668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:25.929872 24668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:25.931104 24668 hybrid_clock.cc:648] HybridClock initialized: now 1786149745931092 us; error 57 us; skew 500 ppm
Aug 08 00:42:25 dist-test-slave-kvfs krb5kdc[24646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149745, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:25.939756 24668 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:25.940423 24668 webserver.cc:460] Webserver started at http://127.20.105.126:45357/ using document root <none> and password file <none>
I20260808 00:42:25.940683 24668 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:25.940734 24668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:25.940850 24668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:25.942133 24668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "3882cc35daf04557b2c0eaca16cc19d7"
format_stamp: "Formatted at 2026-08-08 00:42:25 on dist-test-slave-kvfs"
server_key: "bbbcc631cb608aca6e89f4aa3261543f"
server_key_iv: "85452076e66c87ecc7e7760383d1e32a"
server_key_version: "encryptionkey@0"
I20260808 00:42:25.942517 24668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "3882cc35daf04557b2c0eaca16cc19d7"
format_stamp: "Formatted at 2026-08-08 00:42:25 on dist-test-slave-kvfs"
server_key: "bbbcc631cb608aca6e89f4aa3261543f"
server_key_iv: "85452076e66c87ecc7e7760383d1e32a"
server_key_version: "encryptionkey@0"
I20260808 00:42:25.944416 24668 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:42:25.945326 24688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:25.945559 24668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:25.945664 24668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "3882cc35daf04557b2c0eaca16cc19d7"
format_stamp: "Formatted at 2026-08-08 00:42:25 on dist-test-slave-kvfs"
server_key: "bbbcc631cb608aca6e89f4aa3261543f"
server_key_iv: "85452076e66c87ecc7e7760383d1e32a"
server_key_version: "encryptionkey@0"
I20260808 00:42:25.945736 24668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:25.992065 24668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:25.995087 24668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:25.995256 24668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:26.001133 24668 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:36461
I20260808 00:42:26.001262 24770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:36461 every 8 connection(s)
I20260808 00:42:26.001713 24668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:26.002915 24771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:26.006109 24771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7: Bootstrap starting.
I20260808 00:42:26.006946 24771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:26.007332 24771 log.cc:824] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:26.008388 24771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7: No bootstrap required, opened a new log
I20260808 00:42:26.010339 24771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3882cc35daf04557b2c0eaca16cc19d7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36461 } }
I20260808 00:42:26.010521 24771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:26.010548 24771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3882cc35daf04557b2c0eaca16cc19d7, State: Initialized, Role: FOLLOWER
I20260808 00:42:26.010779 24771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3882cc35daf04557b2c0eaca16cc19d7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36461 } }
I20260808 00:42:26.010859 24771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:26.010885 24771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:26.010959 24771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:26.011540 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 24668
I20260808 00:42:26.011663 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:26.011849 24771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3882cc35daf04557b2c0eaca16cc19d7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36461 } }
I20260808 00:42:26.011883 20901 external_mini_cluster.cc:1428] Setting key 9196ec1be14aa0e044a3de80184b7e15
I20260808 00:42:26.011979 24771 leader_election.cc:302] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3882cc35daf04557b2c0eaca16cc19d7; no voters:
I20260808 00:42:26.012240 24771 leader_election.cc:288] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:26.012269 24776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:26.012423 24776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [term 1 LEADER]: Becoming Leader. State: Replica: 3882cc35daf04557b2c0eaca16cc19d7, State: Running, Role: LEADER
I20260808 00:42:26.012523 24776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3882cc35daf04557b2c0eaca16cc19d7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36461 } }
I20260808 00:42:26.012606 24771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:26.012950 24777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3882cc35daf04557b2c0eaca16cc19d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3882cc35daf04557b2c0eaca16cc19d7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36461 } } }
I20260808 00:42:26.012934 24776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3882cc35daf04557b2c0eaca16cc19d7. Latest consensus state: current_term: 1 leader_uuid: "3882cc35daf04557b2c0eaca16cc19d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3882cc35daf04557b2c0eaca16cc19d7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36461 } } }
I20260808 00:42:26.013131 24777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:26.013142 24776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:26.013386 24787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:26.014281 24787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:26.016314 24787 catalog_manager.cc:1347] Generated new cluster ID: 8ecd5974ecc341ae9f6fe6cf55c8ca31
I20260808 00:42:26.016372 24787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[24646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149745, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:26.026952 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:26.012840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53140 (local address 127.20.105.126:36461)
0808 00:42:26.013045 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:26.013058 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:26.013365 (+ 307us) server_negotiation.cc:407] Connection header received
0808 00:42:26.013640 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:26.013651 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:26.013735 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:26.013906 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:26.015226 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.016747 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.017890 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.018597 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.018723 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:26.022001 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:26.022006 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:26.022007 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:26.022024 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:26.024967 (+ 2943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:26.025447 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:26.025450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:26.025450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:26.025585 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:26.025824 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:26.025824 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:26.025825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:26.026197 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:26.026373 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:26.026499 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 00:42:26.026660 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260808 00:42:26.049700 24787 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:26.050693 24787 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:26.062822 24787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3882cc35daf04557b2c0eaca16cc19d7: Generated new TSK 0
I20260808 00:42:26.063138 24787 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:26.107460 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:36461
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:26.227041 24811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:26.227352 24811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:26.227396 24811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:26.228693 24811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:26.228767 24811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:26.228812 24811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:26.230401 24811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:36461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24811
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:26.230754 24811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:26.231041 24811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:26.237442 24828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:26.237416 24831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:26.237398 24823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:26.237703 24811 server_base.cc:1034] running on GCE node
I20260808 00:42:26.238060 24811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:26.238379 24811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:26.239562 24811 hybrid_clock.cc:648] HybridClock initialized: now 1786149746239538 us; error 58 us; skew 500 ppm
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[24646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149746, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:26.242759 24811 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:26.243348 24811 webserver.cc:460] Webserver started at http://127.20.105.65:41827/ using document root <none> and password file <none>
I20260808 00:42:26.243638 24811 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:26.243713 24811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:26.243880 24811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:26.245255 24811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "cc6d3593913647ad9618e969a6a1b131"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "10d5e3bc54638361e9ef7b733d9522c9"
server_key_iv: "691ff523734117dd0d9ab5f7a5b4430b"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.245689 24811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "cc6d3593913647ad9618e969a6a1b131"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "10d5e3bc54638361e9ef7b733d9522c9"
server_key_iv: "691ff523734117dd0d9ab5f7a5b4430b"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.247378 24811 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:42:26.248404 24840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.248665 24811 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:26.248799 24811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "cc6d3593913647ad9618e969a6a1b131"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "10d5e3bc54638361e9ef7b733d9522c9"
server_key_iv: "691ff523734117dd0d9ab5f7a5b4430b"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.248910 24811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:26.269230 24811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:26.272303 24811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:26.272519 24811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:26.272855 24811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:26.273299 24811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:26.273361 24811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.273412 24811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:26.273438 24811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.281909 24811 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:37811
I20260808 00:42:26.281953 25014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:37811 every 8 connection(s)
I20260808 00:42:26.282478 24811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:26.287246 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 24811
I20260808 00:42:26.287362 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:26.287568 20901 external_mini_cluster.cc:1428] Setting key 3affc9967e49a94bc3c5515917bf08e3
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[24646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149746, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:26.294706 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:26.283556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:41449 (local address 127.20.105.126:36461)
0808 00:42:26.283653 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:26.283656 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:26.283916 (+ 260us) server_negotiation.cc:407] Connection header received
0808 00:42:26.284621 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:26.284623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:26.284636 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:26.284696 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:26.285857 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.286434 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.287892 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.288253 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.288350 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:26.291155 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:26.291157 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:26.291157 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:26.291167 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:26.292988 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:26.293560 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:26.293562 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:26.293562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:26.293627 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:26.293942 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:26.293943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:26.293943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:26.294100 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:26.294157 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:26.294526 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 00:42:26.294621 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:42:26.295097 25017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:26.283518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:36461 (local address 127.20.105.65:41449)
0808 00:42:26.283756 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:26.283806 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:26.284492 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:26.284765 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:26.284770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:26.285481 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:26.285748 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:26.285754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.286541 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.286542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:26.287792 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:26.287797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.288373 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.288374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:26.288544 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:26.289208 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:26.289222 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:26.291053 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:26.293088 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:26.293097 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:26.293100 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:26.293462 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:26.293697 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:26.293698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:26.293699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:26.293864 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:26.294169 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:26.294176 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:26.294374 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 00:42:26.294503 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 00:42:26.294711 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":133,"threads_started":1}
I20260808 00:42:26.295868 25015 heartbeater.cc:347] Connected to a master server at 127.20.105.126:36461
I20260808 00:42:26.296072 25015 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:26.296334 25015 heartbeater.cc:511] Master 127.20.105.126:36461 requested a full tablet report, sending...
I20260808 00:42:26.297132 24720 ts_manager.cc:194] Registered new tserver with Master: cc6d3593913647ad9618e969a6a1b131 (127.20.105.65:37811)
I20260808 00:42:26.297887 24720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:41449
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:26.325791 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:36461
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:26.426295 25021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:26.426563 25021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:26.426594 25021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:26.427970 25021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:26.428057 25021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:26.428089 25021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:26.429639 25021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:36461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:26.429880 25021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:26.430124 25021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:26.436473 25031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:26.436484 25030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:26.436484 25033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:26.436820 25021 server_base.cc:1034] running on GCE node
I20260808 00:42:26.437170 25021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:26.437534 25021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:26.438721 25021 hybrid_clock.cc:648] HybridClock initialized: now 1786149746438701 us; error 37 us; skew 500 ppm
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[24646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149746, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:26.442551 25021 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:26.443177 25021 webserver.cc:460] Webserver started at http://127.20.105.66:44447/ using document root <none> and password file <none>
I20260808 00:42:26.443495 25021 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:26.443573 25021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:26.443738 25021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:26.445071 25021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "044518be5817411f9455d666636bbb48"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "4623d1c3a1cc3263cb26fbbbd02b8f71"
server_key_iv: "b11ca8bc27b2adb71964734fe4b1180b"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.445519 25021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "044518be5817411f9455d666636bbb48"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "4623d1c3a1cc3263cb26fbbbd02b8f71"
server_key_iv: "b11ca8bc27b2adb71964734fe4b1180b"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.447142 25021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:42:26.448110 25042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.448372 25021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:26.448490 25021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "044518be5817411f9455d666636bbb48"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "4623d1c3a1cc3263cb26fbbbd02b8f71"
server_key_iv: "b11ca8bc27b2adb71964734fe4b1180b"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.448582 25021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:26.457119 25021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:26.459880 25021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:26.460110 25021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:26.460394 25021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:26.460742 25021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:26.460779 25021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.460810 25021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:26.460832 25021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.468626 25021 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:40125
I20260808 00:42:26.468678 25214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:40125 every 8 connection(s)
I20260808 00:42:26.469103 25021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:26.474000 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 25021
I20260808 00:42:26.474139 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:26.474367 20901 external_mini_cluster.cc:1428] Setting key 6c09fbe98be61849e10cd191fa01a55b
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[24646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149746, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:26.480563 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:26.470034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:52939 (local address 127.20.105.126:36461)
0808 00:42:26.470152 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:26.470155 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:26.470434 (+ 279us) server_negotiation.cc:407] Connection header received
0808 00:42:26.471057 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:26.471058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:26.471075 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:26.471133 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:26.472317 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.472889 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.474032 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.474363 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.474446 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:26.477139 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:26.477142 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:26.477143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:26.477155 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:26.478864 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:26.479345 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:26.479347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:26.479348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:26.479425 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:26.479703 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:26.479705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:26.479705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:26.479853 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:26.479927 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:26.480338 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 00:42:26.480452 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:42:26.480815 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:26.469978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:36461 (local address 127.20.105.66:52939)
0808 00:42:26.470219 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:26.470309 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:26.470929 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:26.471207 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:26.471211 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:26.471896 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:26.472228 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:26.472235 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.473002 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.473004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:26.473939 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:26.473942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.474471 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.474472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:26.474635 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:26.475349 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:26.475360 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:26.477030 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:26.478981 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:26.478989 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:26.478992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:26.479261 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:26.479502 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:26.479503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:26.479503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:26.479628 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:26.479968 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:26.479975 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:26.480190 (+ 215us) client_negotiation.cc:769] Sending connection context
0808 00:42:26.480312 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 00:42:26.480516 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":126,"threads_started":1}
I20260808 00:42:26.481410 25216 heartbeater.cc:347] Connected to a master server at 127.20.105.126:36461
I20260808 00:42:26.481568 25216 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:26.481843 25216 heartbeater.cc:511] Master 127.20.105.126:36461 requested a full tablet report, sending...
I20260808 00:42:26.482355 24720 ts_manager.cc:194] Registered new tserver with Master: 044518be5817411f9455d666636bbb48 (127.20.105.66:40125)
I20260808 00:42:26.483008 24720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:52939
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:26.507675 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:36461
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:26.606146 25225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:26.606455 25225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:26.606499 25225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:26.607787 25225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:26.607854 25225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:26.607899 25225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:26.609545 25225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:36461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:26.609865 25225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:26.610159 25225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:26.616545 25232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:26.616547 25235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:26.617003 25225 server_base.cc:1034] running on GCE node
W20260808 00:42:26.617069 25240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:26.617336 25225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:26.617663 25225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:26.618865 25225 hybrid_clock.cc:648] HybridClock initialized: now 1786149746618852 us; error 53 us; skew 500 ppm
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[24646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149746, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:26.622150 25225 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:26.622761 25225 webserver.cc:460] Webserver started at http://127.20.105.67:44281/ using document root <none> and password file <none>
I20260808 00:42:26.623078 25225 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:26.623155 25225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:26.623327 25225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:26.624800 25225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "4126b33b6b19445c8faf9cbdcd8cf727"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "3f56144fa7fecf1bea6c1a40c7539925"
server_key_iv: "6001a2480274b261842af7ac2c590bf3"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.625262 25225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "4126b33b6b19445c8faf9cbdcd8cf727"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "3f56144fa7fecf1bea6c1a40c7539925"
server_key_iv: "6001a2480274b261842af7ac2c590bf3"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.626961 25225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:26.627936 25250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.628206 25225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:26.628311 25225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "4126b33b6b19445c8faf9cbdcd8cf727"
format_stamp: "Formatted at 2026-08-08 00:42:26 on dist-test-slave-kvfs"
server_key: "3f56144fa7fecf1bea6c1a40c7539925"
server_key_iv: "6001a2480274b261842af7ac2c590bf3"
server_key_version: "encryptionkey@0"
I20260808 00:42:26.628396 25225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:26.658293 25225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:26.660840 25225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:26.661031 25225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:26.661346 25225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:26.661746 25225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:26.661803 25225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.661851 25225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:26.661880 25225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:26.669059 25225 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:36515
I20260808 00:42:26.669121 25431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:36515 every 8 connection(s)
I20260808 00:42:26.669565 25225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:26.675511 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 25225
I20260808 00:42:26.675671 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:26.676090 20901 external_mini_cluster.cc:1428] Setting key 157c3e658dd4e531c046306aed79b30f
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[24646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149746, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:26.681238 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:26.670500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:45603 (local address 127.20.105.126:36461)
0808 00:42:26.670584 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:26.670586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:26.670796 (+ 210us) server_negotiation.cc:407] Connection header received
0808 00:42:26.671393 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:26.671394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:26.671406 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:26.671480 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:26.672707 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.673329 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.674453 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.674682 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.674762 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:26.677962 (+ 3200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:26.677964 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:26.677964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:26.677974 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:26.679514 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:26.680053 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:26.680055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:26.680055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:26.680133 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:26.680433 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:26.680435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:26.680435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:26.680586 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:26.680669 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:26.681021 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 00:42:26.681126 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:42:26.681490 25437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:26.670455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:36461 (local address 127.20.105.67:45603)
0808 00:42:26.670643 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:26.670695 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:26.671298 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:26.671555 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:26.671559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:26.672317 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:26.672584 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:26.672591 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.673430 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.673432 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:26.674350 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:26.674353 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:26.674793 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:26.674795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:26.674934 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:26.675722 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:26.675736 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:26.677818 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:26.679637 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:26.679647 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:26.679651 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:26.679952 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:26.680218 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:26.680220 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:26.680220 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:26.680361 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:26.680657 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:26.680666 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:26.680855 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 00:42:26.680982 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 00:42:26.681192 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260808 00:42:26.682098 25432 heartbeater.cc:347] Connected to a master server at 127.20.105.126:36461
I20260808 00:42:26.682252 25432 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:26.682489 25432 heartbeater.cc:511] Master 127.20.105.126:36461 requested a full tablet report, sending...
I20260808 00:42:26.683034 24720 ts_manager.cc:194] Registered new tserver with Master: 4126b33b6b19445c8faf9cbdcd8cf727 (127.20.105.67:36515)
I20260808 00:42:26.683487 24720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:45603
I20260808 00:42:26.688349 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:42:26.706619 24785 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:26.705164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.20.105.126:36461)
0808 00:42:26.705250 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:26.705253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:26.705520 (+ 267us) server_negotiation.cc:407] Connection header received
0808 00:42:26.705570 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:26.705571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:26.705584 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:26.705649 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:26.706486 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53144: BlockingRecv error: recv got EOF from 127.0.0.1:53144 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:42:26.707541 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 24811
I20260808 00:42:26.726950 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 25021
I20260808 00:42:26.745615 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 25225
I20260808 00:42:26.762564 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 24668
2026-08-08T00:42:26Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2416 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[25456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[25456](info): set up 2 sockets
Aug 08 00:42:26 dist-test-slave-kvfs krb5kdc[25456](info): commencing operation
krb5kdc: starting...
W20260808 00:42:28.093489 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:28 dist-test-slave-kvfs krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:28.252106 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:38365
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:38365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:28.361125 25474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:28.361470 25474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:28.361505 25474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:28.363333 25474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:28.363417 25474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:28.363433 25474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:28.363447 25474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:28.363461 25474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:28.365587 25474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:38365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:38365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:28.365983 25474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:28.366305 25474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:28.372370 25483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:28.372370 25480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:28.372408 25486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:28.372612 25474 server_base.cc:1034] running on GCE node
I20260808 00:42:28.373083 25474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:28.373458 25474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:28.374637 25474 hybrid_clock.cc:648] HybridClock initialized: now 1786149748374630 us; error 51 us; skew 500 ppm
Aug 08 00:42:28 dist-test-slave-kvfs krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149748, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:28.378203 25474 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:28.378824 25474 webserver.cc:460] Webserver started at http://127.20.105.126:39351/ using document root <none> and password file <none>
I20260808 00:42:28.379109 25474 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:28.379185 25474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:28.379354 25474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:28.380858 25474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13"
format_stamp: "Formatted at 2026-08-08 00:42:28 on dist-test-slave-kvfs"
server_key: "ab45c27397a679816bd5beff61e245a8"
server_key_iv: "7a143e056df54cc7d6539de3e1aeb4e1"
server_key_version: "encryptionkey@0"
I20260808 00:42:28.381323 25474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13"
format_stamp: "Formatted at 2026-08-08 00:42:28 on dist-test-slave-kvfs"
server_key: "ab45c27397a679816bd5beff61e245a8"
server_key_iv: "7a143e056df54cc7d6539de3e1aeb4e1"
server_key_version: "encryptionkey@0"
I20260808 00:42:28.382948 25474 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:28.383847 25496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:28.384104 25474 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:28.384236 25474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13"
format_stamp: "Formatted at 2026-08-08 00:42:28 on dist-test-slave-kvfs"
server_key: "ab45c27397a679816bd5beff61e245a8"
server_key_iv: "7a143e056df54cc7d6539de3e1aeb4e1"
server_key_version: "encryptionkey@0"
I20260808 00:42:28.384330 25474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:28.406482 25474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:28.409627 25474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:28.409780 25474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:28.415135 25474 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:38365
I20260808 00:42:28.415174 25575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:38365 every 8 connection(s)
I20260808 00:42:28.416100 25474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:28.417088 25576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:28.419998 25576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13: Bootstrap starting.
I20260808 00:42:28.420809 25576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:28.421123 25576 log.cc:824] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:28.422098 25576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13: No bootstrap required, opened a new log
I20260808 00:42:28.422067 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 25474
I20260808 00:42:28.422209 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:28.422475 20901 external_mini_cluster.cc:1428] Setting key 816fe859bd8c53ab41ff94d54bc86f82
I20260808 00:42:28.425571 25576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38365 } }
I20260808 00:42:28.425803 25576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:28.425848 25576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d2b0bfb0f1c49009e3cbdc0f8a09d13, State: Initialized, Role: FOLLOWER
I20260808 00:42:28.426011 25576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38365 } }
I20260808 00:42:28.426112 25576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:28.426149 25576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:28.426206 25576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:28.427165 25576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38365 } }
I20260808 00:42:28.427320 25576 leader_election.cc:302] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d2b0bfb0f1c49009e3cbdc0f8a09d13; no voters:
I20260808 00:42:28.427583 25576 leader_election.cc:288] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:28.427619 25587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:28.427770 25587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [term 1 LEADER]: Becoming Leader. State: Replica: 0d2b0bfb0f1c49009e3cbdc0f8a09d13, State: Running, Role: LEADER
I20260808 00:42:28.427955 25587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38365 } }
I20260808 00:42:28.428051 25576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:28.428440 25587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d2b0bfb0f1c49009e3cbdc0f8a09d13. Latest consensus state: current_term: 1 leader_uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38365 } } }
I20260808 00:42:28.428674 25587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:28.429312 25598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:28.430154 25588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d2b0bfb0f1c49009e3cbdc0f8a09d13" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38365 } } }
I20260808 00:42:28.430253 25588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:28.430569 25598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:42:28 dist-test-slave-kvfs krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149748, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:28.432547 25598 catalog_manager.cc:1347] Generated new cluster ID: c750d82c0c47407ab9dc5cd45495ec64
I20260808 00:42:28.432610 25598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:28.438028 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:28.423161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.20.105.126:38365)
0808 00:42:28.423442 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:28.423454 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:28.423496 (+ 42us) server_negotiation.cc:407] Connection header received
0808 00:42:28.424446 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:28.424460 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:28.424545 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:28.424758 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:28.425516 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:28.427165 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:28.428406 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:28.429137 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:28.429267 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:28.432811 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:28.432816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:28.432816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:28.432831 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:28.435119 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:28.435595 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:28.435599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:28.435599 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:28.435740 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:28.435989 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:28.435990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:28.435990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:28.436394 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:28.436589 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:28.437250 (+ 661us) server_negotiation.cc:299] Negotiation successful
0808 00:42:28.437421 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":129,"threads_started":1}
I20260808 00:42:28.440604 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:42:28.441985 25598 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:28.443042 25598 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:28.453927 25598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d2b0bfb0f1c49009e3cbdc0f8a09d13: Generated new TSK 0
I20260808 00:42:28.454209 25598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:42:28.468859 25585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:28.463976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.20.105.126:38365)
0808 00:42:28.464078 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:28.464082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:28.464198 (+ 116us) server_negotiation.cc:407] Connection header received
0808 00:42:28.464301 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:28.464302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:28.464313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:28.464378 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:28.464962 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:28.465587 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:28.466792 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:28.467093 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:28.467168 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:28.468122 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:28.468124 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:28.468124 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:28.468432 (+ 308us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:28.468459 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:28.468466 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:28.468523 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:28.468639 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 00:42:28.468696 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 00:42:28.472218 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 25474
2026-08-08T00:42:28Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1713 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:28 dist-test-slave-kvfs krb5kdc[25625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:28 dist-test-slave-kvfs krb5kdc[25625](info): set up 2 sockets
Aug 08 00:42:28 dist-test-slave-kvfs krb5kdc[25625](info): commencing operation
krb5kdc: starting...
W20260808 00:42:29.777199 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:29 dist-test-slave-kvfs krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:29.943562 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:43025
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:43025
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:30.043084 25643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:30.043360 25643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:30.043386 25643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:30.044731 25643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:30.044780 25643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:30.044795 25643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:30.044808 25643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:30.044821 25643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:30.046489 25643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:43025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:43025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:30.046814 25643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:30.047089 25643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:30.052799 25653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:30.052812 25651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:30.052819 25655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:30.053329 25643 server_base.cc:1034] running on GCE node
I20260808 00:42:30.053668 25643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:30.054010 25643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:30.055210 25643 hybrid_clock.cc:648] HybridClock initialized: now 1786149750055201 us; error 37 us; skew 500 ppm
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:30.058197 25643 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:30.058760 25643 webserver.cc:460] Webserver started at http://127.20.105.126:45175/ using document root <none> and password file <none>
I20260808 00:42:30.059029 25643 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:30.059104 25643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:30.059259 25643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:30.060581 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "b3b23de625fd44abb9ea4ddd8f6082f7"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "087b104fdf9ceb755bf25f6dd21c0df2"
server_key_iv: "1fdfea105b31bf2695f8e48ecd64bff6"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.061064 25643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "b3b23de625fd44abb9ea4ddd8f6082f7"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "087b104fdf9ceb755bf25f6dd21c0df2"
server_key_iv: "1fdfea105b31bf2695f8e48ecd64bff6"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.062554 25643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:30.063321 25664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.063498 25643 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:42:30.063611 25643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "b3b23de625fd44abb9ea4ddd8f6082f7"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "087b104fdf9ceb755bf25f6dd21c0df2"
server_key_iv: "1fdfea105b31bf2695f8e48ecd64bff6"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.063690 25643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:30.081749 25643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:30.084326 25643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:30.084512 25643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:30.089761 25643 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:43025
I20260808 00:42:30.089802 25748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:43025 every 8 connection(s)
I20260808 00:42:30.090265 25643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:30.091607 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 25643
I20260808 00:42:30.091711 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:30.091866 25749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:30.091945 20901 external_mini_cluster.cc:1428] Setting key 22513a65f5b6c15f71d87547f83627d8
I20260808 00:42:30.096426 25749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7: Bootstrap starting.
I20260808 00:42:30.097327 25749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:30.097715 25749 log.cc:824] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:30.098676 25749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7: No bootstrap required, opened a new log
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:30.101231 25749 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b23de625fd44abb9ea4ddd8f6082f7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43025 } }
I20260808 00:42:30.101397 25749 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:30.101424 25749 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b23de625fd44abb9ea4ddd8f6082f7, State: Initialized, Role: FOLLOWER
I20260808 00:42:30.101614 25749 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b23de625fd44abb9ea4ddd8f6082f7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43025 } }
I20260808 00:42:30.101727 25749 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:30.101774 25749 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:30.101840 25749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:30.102470 25749 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b23de625fd44abb9ea4ddd8f6082f7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43025 } }
I20260808 00:42:30.102609 25749 leader_election.cc:302] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3b23de625fd44abb9ea4ddd8f6082f7; no voters:
I20260808 00:42:30.102814 25749 leader_election.cc:288] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:30.102876 25757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:30.103063 25757 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [term 1 LEADER]: Becoming Leader. State: Replica: b3b23de625fd44abb9ea4ddd8f6082f7, State: Running, Role: LEADER
I20260808 00:42:30.103133 25749 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:30.103171 25757 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b23de625fd44abb9ea4ddd8f6082f7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43025 } }
I20260808 00:42:30.103580 25760 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3b23de625fd44abb9ea4ddd8f6082f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b23de625fd44abb9ea4ddd8f6082f7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43025 } } }
I20260808 00:42:30.103806 25760 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:30.104206 25769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:30.103569 25757 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3b23de625fd44abb9ea4ddd8f6082f7. Latest consensus state: current_term: 1 leader_uuid: "b3b23de625fd44abb9ea4ddd8f6082f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3b23de625fd44abb9ea4ddd8f6082f7" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43025 } } }
I20260808 00:42:30.104508 25757 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:30.105254 25769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:30.106174 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:30.092670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.20.105.126:43025)
0808 00:42:30.092895 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:30.092907 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:30.092951 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:42:30.093506 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:30.093518 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:30.093596 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:30.093785 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:30.094456 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.096103 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.096984 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.097742 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.097877 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:30.101039 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:30.101044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:30.101044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:30.101058 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:30.103668 (+ 2610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.104275 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.104279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.104279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.104414 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.104710 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.104711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.104712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.105102 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:30.105329 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:30.105470 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 00:42:30.105793 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":92,"threads_started":1}
I20260808 00:42:30.107312 25769 catalog_manager.cc:1347] Generated new cluster ID: 72796d28d2054500ab1b61e258798849
I20260808 00:42:30.107357 25769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:30.133633 25769 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:30.134336 25769 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:30.140555 25769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3b23de625fd44abb9ea4ddd8f6082f7: Generated new TSK 0
I20260808 00:42:30.140798 25769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:30.187397 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43025
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:30.292198 25789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:30.292515 25789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:30.292559 25789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:30.293812 25789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:30.293884 25789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:30.293936 25789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:30.295673 25789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:30.295981 25789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:30.296300 25789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:30.302487 25795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:30.302562 25789 server_base.cc:1034] running on GCE node
W20260808 00:42:30.302487 25794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:30.302506 25798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:30.303035 25789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:30.303326 25789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:30.304497 25789 hybrid_clock.cc:648] HybridClock initialized: now 1786149750304487 us; error 41 us; skew 500 ppm
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:30.307538 25789 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:30.308204 25789 webserver.cc:460] Webserver started at http://127.20.105.65:33371/ using document root <none> and password file <none>
I20260808 00:42:30.308478 25789 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:30.308547 25789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:30.308720 25789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:30.310016 25789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "9d2c5b4222014905878f70ba9ebad437"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "3a4cac67e7184174d83a09ce33f3c649"
server_key_iv: "e3cfafc033f0fd4fb9f18fad87a5d3a0"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.310461 25789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "9d2c5b4222014905878f70ba9ebad437"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "3a4cac67e7184174d83a09ce33f3c649"
server_key_iv: "e3cfafc033f0fd4fb9f18fad87a5d3a0"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.312255 25789 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:42:30.313208 25811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.313409 25789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:30.313512 25789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "9d2c5b4222014905878f70ba9ebad437"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "3a4cac67e7184174d83a09ce33f3c649"
server_key_iv: "e3cfafc033f0fd4fb9f18fad87a5d3a0"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.313609 25789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:30.329694 25789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:30.332578 25789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:30.332769 25789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:30.333091 25789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:30.333483 25789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:30.333525 25789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.333555 25789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:30.333570 25789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.340988 25789 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:41597
I20260808 00:42:30.341081 25976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:41597 every 8 connection(s)
I20260808 00:42:30.341478 25789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:30.346315 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 25789
I20260808 00:42:30.346469 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:30.346714 20901 external_mini_cluster.cc:1428] Setting key 1066864dcd326b5ef21023e419d9ec63
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:30.354048 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:30.342450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:52241 (local address 127.20.105.126:43025)
0808 00:42:30.342547 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:30.342550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:30.342807 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:42:30.343433 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:30.343434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:30.343447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:30.343519 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:30.344731 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.345331 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.346823 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.347197 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.347279 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:30.350375 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:30.350377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:30.350378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:30.350388 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:30.352297 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.352757 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.352759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.352760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.352833 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.353172 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.353173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.353173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.353345 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:30.353418 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:30.353750 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 00:42:30.353872 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 00:42:30.354250 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:30.342426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43025 (local address 127.20.105.65:52241)
0808 00:42:30.342640 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:30.342695 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:30.343329 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:30.343604 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:30.343608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:30.344360 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:30.344619 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:30.344625 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.345427 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.345428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:30.346704 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:30.346707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.347287 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.347288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:30.347424 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:30.348097 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:30.348110 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:30.350262 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:30.352398 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:30.352407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:30.352409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:30.352675 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:30.352958 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:30.352959 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:30.352959 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:30.353097 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:30.353420 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:30.353428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:30.353586 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 00:42:30.353695 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 00:42:30.353859 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":122,"threads_started":1}
I20260808 00:42:30.355101 25977 heartbeater.cc:347] Connected to a master server at 127.20.105.126:43025
I20260808 00:42:30.355237 25977 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:30.355502 25977 heartbeater.cc:511] Master 127.20.105.126:43025 requested a full tablet report, sending...
I20260808 00:42:30.356346 25696 ts_manager.cc:194] Registered new tserver with Master: 9d2c5b4222014905878f70ba9ebad437 (127.20.105.65:41597)
I20260808 00:42:30.357429 25696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:52241
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:30.383612 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43025
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:30.479660 25982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:30.479911 25982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:30.479936 25982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:30.481213 25982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:30.481261 25982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:30.481288 25982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:30.482821 25982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:30.483098 25982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:30.483371 25982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:30.489223 25992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:30.489413 25990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:30.489470 25982 server_base.cc:1034] running on GCE node
W20260808 00:42:30.489216 25988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:30.489753 25982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:30.490063 25982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:30.491240 25982 hybrid_clock.cc:648] HybridClock initialized: now 1786149750491226 us; error 41 us; skew 500 ppm
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:30.494174 25982 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:30.494715 25982 webserver.cc:460] Webserver started at http://127.20.105.66:38317/ using document root <none> and password file <none>
I20260808 00:42:30.494930 25982 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:30.494972 25982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:30.495080 25982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:30.496331 25982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "5c7e10b12dfd4e0c9ac51672919df35a"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "1ad13b865af25ae40447e84b0d4a39de"
server_key_iv: "120dfb169c40791f5b4243485b3a07ba"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.496685 25982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "5c7e10b12dfd4e0c9ac51672919df35a"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "1ad13b865af25ae40447e84b0d4a39de"
server_key_iv: "120dfb169c40791f5b4243485b3a07ba"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.498389 25982 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:30.499369 26005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.499588 25982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:30.499691 25982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "5c7e10b12dfd4e0c9ac51672919df35a"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "1ad13b865af25ae40447e84b0d4a39de"
server_key_iv: "120dfb169c40791f5b4243485b3a07ba"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.499758 25982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:30.513969 25982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:30.516664 25982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:30.516888 25982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:30.517175 25982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:30.517515 25982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:30.517550 25982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.517578 25982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:30.517627 25982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.524984 25982 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:35329
I20260808 00:42:30.525068 26173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:35329 every 8 connection(s)
I20260808 00:42:30.525450 25982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:30.531551 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 25982
I20260808 00:42:30.531678 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:30.531925 20901 external_mini_cluster.cc:1428] Setting key 30fb11ac70d870ce2e6dc261276013f4
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:30.537077 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:30.526382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:50135 (local address 127.20.105.126:43025)
0808 00:42:30.526461 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:30.526463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:30.526735 (+ 272us) server_negotiation.cc:407] Connection header received
0808 00:42:30.527383 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:30.527384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:30.527398 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:30.527467 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:30.528706 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.529308 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.530482 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.530706 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.530784 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:30.533662 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:30.533663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:30.533664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:30.533673 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:30.535380 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.535849 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.535850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.535851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.535924 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.536248 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.536249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.536249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.536427 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:30.536506 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:30.536813 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 00:42:30.536928 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:42:30.537173 26177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:30.526360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43025 (local address 127.20.105.66:50135)
0808 00:42:30.526552 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:30.526603 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:30.527231 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:30.527565 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:30.527570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:30.528334 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:30.528602 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:30.528608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.529433 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.529434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:30.530386 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:30.530388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.530809 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.530810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:30.530946 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:30.531635 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:30.531651 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:30.533552 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:30.535480 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:30.535489 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:30.535492 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:30.535770 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:30.535993 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:30.535994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:30.535994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:30.536181 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:30.536498 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:30.536504 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:30.536660 (+ 156us) client_negotiation.cc:769] Sending connection context
0808 00:42:30.536751 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 00:42:30.536911 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260808 00:42:30.537755 26175 heartbeater.cc:347] Connected to a master server at 127.20.105.126:43025
I20260808 00:42:30.537894 26175 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:30.538132 26175 heartbeater.cc:511] Master 127.20.105.126:43025 requested a full tablet report, sending...
I20260808 00:42:30.538601 25696 ts_manager.cc:194] Registered new tserver with Master: 5c7e10b12dfd4e0c9ac51672919df35a (127.20.105.66:35329)
I20260808 00:42:30.539232 25696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:50135
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:30.567835 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43025
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:38615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:30.669041 26181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:30.669354 26181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:30.669397 26181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:30.670656 26181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:30.670727 26181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:30.670773 26181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:30.672487 26181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:38615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43025
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:30.672811 26181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:30.673106 26181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:30.679289 26188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:30.679363 26190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:30.679394 26187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:30.679447 26181 server_base.cc:1034] running on GCE node
I20260808 00:42:30.679742 26181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:30.680066 26181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:30.681255 26181 hybrid_clock.cc:648] HybridClock initialized: now 1786149750681233 us; error 41 us; skew 500 ppm
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:30.684499 26181 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:30.685077 26181 webserver.cc:460] Webserver started at http://127.20.105.67:35181/ using document root <none> and password file <none>
I20260808 00:42:30.685370 26181 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:30.685446 26181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:30.685627 26181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:30.686885 26181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "781aa89891654641a664207aaa34c142"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "23f62f6470db109bac7f5841fb53a8f0"
server_key_iv: "8ed0501468fd9aff286081f5f1572cff"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.687314 26181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "781aa89891654641a664207aaa34c142"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "23f62f6470db109bac7f5841fb53a8f0"
server_key_iv: "8ed0501468fd9aff286081f5f1572cff"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.688943 26181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 00:42:30.689810 26200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.690034 26181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:30.690148 26181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "781aa89891654641a664207aaa34c142"
format_stamp: "Formatted at 2026-08-08 00:42:30 on dist-test-slave-kvfs"
server_key: "23f62f6470db109bac7f5841fb53a8f0"
server_key_iv: "8ed0501468fd9aff286081f5f1572cff"
server_key_version: "encryptionkey@0"
I20260808 00:42:30.690227 26181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:30.701126 26181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:30.703408 26181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:30.703591 26181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:30.703847 26181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:30.704344 26181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:30.704412 26181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.704461 26181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:30.704512 26181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:30.711532 26181 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:41839
I20260808 00:42:30.711617 26368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:41839 every 8 connection(s)
I20260808 00:42:30.712055 26181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:30.716270 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 26181
I20260808 00:42:30.716365 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:30.716566 20901 external_mini_cluster.cc:1428] Setting key 09dc054e5af13ab18655726bd17982da
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:30.722787 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:30.712965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:41205 (local address 127.20.105.126:43025)
0808 00:42:30.713032 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:30.713035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:30.713251 (+ 216us) server_negotiation.cc:407] Connection header received
0808 00:42:30.713946 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:30.713948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:30.713961 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:30.714029 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:30.715163 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.715745 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.717115 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.717423 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.717514 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:30.719978 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:30.719979 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:30.719980 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:30.719988 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:30.721322 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.721749 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.721751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.721751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.721816 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.722070 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.722071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.722071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.722207 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:30.722284 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:30.722583 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 00:42:30.722679 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:42:30.723043 26371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:30.712943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43025 (local address 127.20.105.67:41205)
0808 00:42:30.713125 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:30.713168 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:30.713781 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:30.714122 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:30.714127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:30.714823 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:30.715069 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:30.715074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.715829 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.715830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:30.717015 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:30.717036 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.717524 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.717525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:30.717693 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:30.718323 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:30.718333 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:30.719871 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:30.721407 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:30.721414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:30.721417 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:30.721664 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:30.721881 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:30.721881 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:30.721882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:30.722000 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:30.722272 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:30.722279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:30.722433 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 00:42:30.722561 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 00:42:30.722745 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260808 00:42:30.723657 26369 heartbeater.cc:347] Connected to a master server at 127.20.105.126:43025
I20260808 00:42:30.723773 26369 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:30.723982 26369 heartbeater.cc:511] Master 127.20.105.126:43025 requested a full tablet report, sending...
I20260808 00:42:30.724465 25696 ts_manager.cc:194] Registered new tserver with Master: 781aa89891654641a664207aaa34c142 (127.20.105.67:41839)
I20260808 00:42:30.724887 25696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:41205
I20260808 00:42:30.728502 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[25625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149750, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:30.757306 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:30.749412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53712 (local address 127.20.105.126:43025)
0808 00:42:30.749508 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:30.749510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:30.749590 (+ 80us) server_negotiation.cc:407] Connection header received
0808 00:42:30.749665 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:30.749666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:30.749677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:30.749743 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:30.750394 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.750983 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.751779 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:30.751988 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:30.752104 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:30.754731 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:30.754733 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:30.754733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:30.754743 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:30.756136 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.756478 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.756479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.756479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.756537 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:30.756704 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:30.756704 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:30.756705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:30.756882 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:30.756972 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:30.757119 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 00:42:30.757191 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 00:42:30.757653 25696 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53712
I20260808 00:42:30.758221 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 25789
I20260808 00:42:30.778622 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 25982
I20260808 00:42:30.794411 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 26181
I20260808 00:42:30.808894 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 25643
2026-08-08T00:42:30Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2334 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[26384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[26384](info): set up 2 sockets
Aug 08 00:42:30 dist-test-slave-kvfs krb5kdc[26384](info): commencing operation
krb5kdc: starting...
W20260808 00:42:32.083269 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:32.228101 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:41969
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:41969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:32.325469 26402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:32.325796 26402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:32.325842 26402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:32.327205 26402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:32.327281 26402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:32.327304 26402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:32.327324 26402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:32.327355 26402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:32.329264 26402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:41969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:32.329628 26402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:32.329918 26402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:32.335991 26411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:32.335934 26410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:32.335944 26413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:32.336131 26402 server_base.cc:1034] running on GCE node
I20260808 00:42:32.336609 26402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:32.337002 26402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:32.338177 26402 hybrid_clock.cc:648] HybridClock initialized: now 1786149752338151 us; error 35 us; skew 500 ppm
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:32.347987 26402 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:32.348682 26402 webserver.cc:460] Webserver started at http://127.20.105.126:44691/ using document root <none> and password file <none>
I20260808 00:42:32.348984 26402 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:32.349041 26402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:32.349208 26402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:32.350524 26402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "73c4b172ede746f49856b11301f15b78"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "86fe24f506fc15f5819c8bd4ce9b2dcb"
server_key_iv: "0494567c3030f0e565a1a4edcb97a2a3"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.350960 26402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "73c4b172ede746f49856b11301f15b78"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "86fe24f506fc15f5819c8bd4ce9b2dcb"
server_key_iv: "0494567c3030f0e565a1a4edcb97a2a3"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.352690 26402 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:42:32.353600 26422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.353837 26402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:32.353940 26402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "73c4b172ede746f49856b11301f15b78"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "86fe24f506fc15f5819c8bd4ce9b2dcb"
server_key_iv: "0494567c3030f0e565a1a4edcb97a2a3"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.354029 26402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:32.366752 26402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:32.369395 26402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:32.369580 26402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:32.374905 26402 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:41969
I20260808 00:42:32.374953 26504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:41969 every 8 connection(s)
I20260808 00:42:32.375602 26402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:32.376595 26506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:32.379539 26506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78: Bootstrap starting.
I20260808 00:42:32.380386 26506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:32.380743 26506 log.cc:824] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:32.381583 26506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78: No bootstrap required, opened a new log
I20260808 00:42:32.383474 26506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c4b172ede746f49856b11301f15b78" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41969 } }
I20260808 00:42:32.383641 26506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:32.383688 26506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73c4b172ede746f49856b11301f15b78, State: Initialized, Role: FOLLOWER
I20260808 00:42:32.383867 26506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c4b172ede746f49856b11301f15b78" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41969 } }
I20260808 00:42:32.383944 26506 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:32.383968 26506 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:32.384075 26506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:32.385103 26506 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c4b172ede746f49856b11301f15b78" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41969 } }
I20260808 00:42:32.385233 26506 leader_election.cc:302] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73c4b172ede746f49856b11301f15b78; no voters:
I20260808 00:42:32.385435 26506 leader_election.cc:288] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:32.385485 26509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:32.385701 26509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [term 1 LEADER]: Becoming Leader. State: Replica: 73c4b172ede746f49856b11301f15b78, State: Running, Role: LEADER
I20260808 00:42:32.385797 26506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:32.385785 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 26402
I20260808 00:42:32.385905 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:32.385840 26509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c4b172ede746f49856b11301f15b78" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41969 } }
I20260808 00:42:32.386147 20901 external_mini_cluster.cc:1428] Setting key acd40edf2cd63fdfabb6a1fee4b107e1
I20260808 00:42:32.386405 26511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73c4b172ede746f49856b11301f15b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c4b172ede746f49856b11301f15b78" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41969 } } }
I20260808 00:42:32.386492 26511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:32.386729 26513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73c4b172ede746f49856b11301f15b78. Latest consensus state: current_term: 1 leader_uuid: "73c4b172ede746f49856b11301f15b78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73c4b172ede746f49856b11301f15b78" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41969 } } }
I20260808 00:42:32.386792 26513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:32.387298 26520 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:32.387962 26520 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:32.389698 26520 catalog_manager.cc:1347] Generated new cluster ID: 34a176a1a74e466ea1b2a5f34f8d5124
I20260808 00:42:32.389755 26520 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:32.393356 26520 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:32.394107 26520 catalog_manager.cc:1504] Loading token signing keys...
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:32.398928 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:32.387103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.20.105.126:41969)
0808 00:42:32.387329 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:32.387341 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:32.387383 (+ 42us) server_negotiation.cc:407] Connection header received
0808 00:42:32.387528 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:32.387549 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:32.387629 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:32.387847 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:32.388889 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.390433 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.391594 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.392330 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.392461 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:32.395318 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:32.395322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:32.395323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:32.395337 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:32.397237 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:32.397645 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:32.397649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:32.397649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:32.397756 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:32.397972 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:32.397973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:32.397974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:32.398237 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:32.398439 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:32.398521 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 00:42:32.398711 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":114,"threads_started":1}
I20260808 00:42:32.400398 26520 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73c4b172ede746f49856b11301f15b78: Generated new TSK 0
I20260808 00:42:32.400604 26520 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:32.437781 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41969
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:32.535398 26543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:32.535715 26543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:32.535760 26543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:32.537149 26543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:32.537225 26543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:32.537272 26543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:32.538980 26543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:32.539479 26543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:32.539837 26543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:32.546356 26551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:32.546510 26543 server_base.cc:1034] running on GCE node
W20260808 00:42:32.546408 26549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:32.546365 26554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:32.546833 26543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:32.547120 26543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:32.548316 26543 hybrid_clock.cc:648] HybridClock initialized: now 1786149752548300 us; error 43 us; skew 500 ppm
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:32.551350 26543 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:32.551890 26543 webserver.cc:460] Webserver started at http://127.20.105.65:36615/ using document root <none> and password file <none>
I20260808 00:42:32.552207 26543 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:32.552281 26543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:32.552449 26543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:32.553704 26543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "d0a7cb5f36b94d909cc98a5ed7c26d2c"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "b1bf04f59f20a85aa89bef94b83563fc"
server_key_iv: "ed9e7133ffee242dac1ac30bd8034914"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.554136 26543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "d0a7cb5f36b94d909cc98a5ed7c26d2c"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "b1bf04f59f20a85aa89bef94b83563fc"
server_key_iv: "ed9e7133ffee242dac1ac30bd8034914"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.555668 26543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:42:32.556587 26565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.556762 26543 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:32.556872 26543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "d0a7cb5f36b94d909cc98a5ed7c26d2c"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "b1bf04f59f20a85aa89bef94b83563fc"
server_key_iv: "ed9e7133ffee242dac1ac30bd8034914"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.556950 26543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:32.569535 26543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:32.572340 26543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:32.572533 26543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:32.572830 26543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:32.573237 26543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:32.573294 26543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.573339 26543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:32.573370 26543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.580406 26543 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:36371
I20260808 00:42:32.580458 26729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:36371 every 8 connection(s)
I20260808 00:42:32.580865 26543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:32.585316 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 26543
I20260808 00:42:32.585469 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:32.585711 20901 external_mini_cluster.cc:1428] Setting key 9b952edfb50a827082b1c5be921f49d6
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:32.593246 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:32.582258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:49031 (local address 127.20.105.126:41969)
0808 00:42:32.582323 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:32.582325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:32.582339 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:42:32.583436 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:32.583438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:32.583455 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:32.583513 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:32.584375 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.584966 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.586327 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.586619 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.586689 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:32.589724 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:32.589727 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:32.589727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:32.589737 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:32.591354 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:32.591899 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:32.591900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:32.591900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:32.591959 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:32.592270 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:32.592271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:32.592272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:32.592450 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:32.592527 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:32.592956 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 00:42:32.593091 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:42:32.593993 26734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:32.581878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41969 (local address 127.20.105.65:49031)
0808 00:42:32.582104 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:32.582165 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:32.583278 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:32.583610 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:32.583615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:32.583946 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:32.584266 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:32.584272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.585067 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.585068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:32.586242 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:32.586246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.586696 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.586698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:32.586849 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:32.587568 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:32.587590 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:32.589614 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:32.591442 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:32.591451 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:32.591453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:32.591821 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:32.592038 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:32.592039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:32.592039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:32.592199 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:32.592533 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:32.592543 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:32.592750 (+ 207us) client_negotiation.cc:769] Sending connection context
0808 00:42:32.593423 (+ 673us) client_negotiation.cc:240] Negotiation successful
0808 00:42:32.593650 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260808 00:42:32.594666 26732 heartbeater.cc:347] Connected to a master server at 127.20.105.126:41969
I20260808 00:42:32.594796 26732 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:32.595021 26732 heartbeater.cc:511] Master 127.20.105.126:41969 requested a full tablet report, sending...
I20260808 00:42:32.595681 26446 ts_manager.cc:194] Registered new tserver with Master: d0a7cb5f36b94d909cc98a5ed7c26d2c (127.20.105.65:36371)
I20260808 00:42:32.596431 26446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:49031
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:32.620328 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41969
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:32.715690 26740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:32.715991 26740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:32.716056 26740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:32.717344 26740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:32.717411 26740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:32.717455 26740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:32.719252 26740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:32.719569 26740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:32.719861 26740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:32.726140 26752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:32.726135 26754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:32.726138 26756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:32.726533 26740 server_base.cc:1034] running on GCE node
I20260808 00:42:32.726760 26740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:32.727054 26740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:32.728219 26740 hybrid_clock.cc:648] HybridClock initialized: now 1786149752728206 us; error 31 us; skew 500 ppm
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:32.731178 26740 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:32.731724 26740 webserver.cc:460] Webserver started at http://127.20.105.66:41153/ using document root <none> and password file <none>
I20260808 00:42:32.732110 26740 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:32.732170 26740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:32.732285 26740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:32.733484 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "1afaf2b9505445e2abcd86309db984ee"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "2ef0a80c01739e38ef16b6e0e2013e3b"
server_key_iv: "533803c89c793e9e147b97d690627115"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.733831 26740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "1afaf2b9505445e2abcd86309db984ee"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "2ef0a80c01739e38ef16b6e0e2013e3b"
server_key_iv: "533803c89c793e9e147b97d690627115"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.735271 26740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:42:32.736095 26766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.736270 26740 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:32.736368 26740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "1afaf2b9505445e2abcd86309db984ee"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "2ef0a80c01739e38ef16b6e0e2013e3b"
server_key_iv: "533803c89c793e9e147b97d690627115"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.736460 26740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:32.756134 26740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:32.758970 26740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:32.759157 26740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:32.759497 26740 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:32.759925 26740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:32.759968 26740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.759999 26740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:32.760011 26740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.767952 26740 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:45777
I20260808 00:42:32.768066 26924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:45777 every 8 connection(s)
I20260808 00:42:32.768504 26740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:32.778195 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 26740
I20260808 00:42:32.778366 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:32.778662 20901 external_mini_cluster.cc:1428] Setting key 04da82262b59b412c53c9ccac82b1411
I20260808 00:42:32.780848 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:32.769556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:33677 (local address 127.20.105.126:41969)
0808 00:42:32.769654 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:32.769657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:32.769867 (+ 210us) server_negotiation.cc:407] Connection header received
0808 00:42:32.770979 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:32.770981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:32.770995 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:32.771069 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:32.771866 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.772553 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.773822 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.774049 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.774129 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:32.777164 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:32.777166 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:32.777167 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:32.777176 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:32.778975 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:32.779506 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:32.779508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:32.779509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:32.779613 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:32.779919 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:32.779921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:32.779922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:32.780152 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:32.780240 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:32.780546 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 00:42:32.780669 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 00:42:32.780892 26927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:32.769490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41969 (local address 127.20.105.66:33677)
0808 00:42:32.769686 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:32.769742 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:32.770823 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:32.771167 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:32.771172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:32.771518 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:32.771765 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:32.771770 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.772686 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.772688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:32.773719 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:32.773723 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.774168 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.774169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:32.774337 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:32.775047 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:32.775058 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:32.777041 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:32.779101 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:32.779112 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:32.779115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:32.779414 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:32.779697 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:32.779698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:32.779698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:32.779840 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:32.780225 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:32.780231 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:32.780394 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 00:42:32.780498 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:42:32.780671 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":105,"threads_started":1}
I20260808 00:42:32.781482 26925 heartbeater.cc:347] Connected to a master server at 127.20.105.126:41969
I20260808 00:42:32.781621 26925 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:32.781854 26925 heartbeater.cc:511] Master 127.20.105.126:41969 requested a full tablet report, sending...
I20260808 00:42:32.782331 26446 ts_manager.cc:194] Registered new tserver with Master: 1afaf2b9505445e2abcd86309db984ee (127.20.105.66:45777)
I20260808 00:42:32.782765 26446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:33677
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:32.815801 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41969
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:32.918911 26932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:32.919216 26932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:32.919243 26932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:32.920578 26932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:32.920634 26932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:32.920662 26932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:32.922775 26932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:32.923121 26932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:32.923416 26932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:32.930387 26940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:32.930389 26937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:32.930626 26932 server_base.cc:1034] running on GCE node
W20260808 00:42:32.930387 26938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:32.930980 26932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:32.931313 26932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:32.932508 26932 hybrid_clock.cc:648] HybridClock initialized: now 1786149752932483 us; error 40 us; skew 500 ppm
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:32.935737 26932 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:32.936367 26932 webserver.cc:460] Webserver started at http://127.20.105.67:36579/ using document root <none> and password file <none>
I20260808 00:42:32.936631 26932 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:32.936677 26932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:32.936800 26932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:32.938042 26932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "4dccddc538c94caebd59c474e9a2bfc6"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "5b00b2b3d6f0b701da682b79b85cdd0c"
server_key_iv: "2c76590fd36405740355c89f0efe4ace"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.938416 26932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "4dccddc538c94caebd59c474e9a2bfc6"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "5b00b2b3d6f0b701da682b79b85cdd0c"
server_key_iv: "2c76590fd36405740355c89f0efe4ace"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.939989 26932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:42:32.941005 26951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.941237 26932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:32.941361 26932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "4dccddc538c94caebd59c474e9a2bfc6"
format_stamp: "Formatted at 2026-08-08 00:42:32 on dist-test-slave-kvfs"
server_key: "5b00b2b3d6f0b701da682b79b85cdd0c"
server_key_iv: "2c76590fd36405740355c89f0efe4ace"
server_key_version: "encryptionkey@0"
I20260808 00:42:32.941453 26932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:32.955381 26932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:32.958127 26932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:32.958328 26932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:32.958642 26932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:32.959044 26932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:32.959108 26932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.959151 26932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:32.959192 26932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:32.966910 26932 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:35901
I20260808 00:42:32.966966 27121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:35901 every 8 connection(s)
I20260808 00:42:32.967401 26932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:32.974644 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 26932
I20260808 00:42:32.974804 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:32.975086 20901 external_mini_cluster.cc:1428] Setting key 712a9899fcda9d2bf04201539276f726
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:32.980149 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:32.968416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:39407 (local address 127.20.105.126:41969)
0808 00:42:32.968480 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:32.968483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:32.968748 (+ 265us) server_negotiation.cc:407] Connection header received
0808 00:42:32.969953 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:32.969955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:32.969975 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:32.970031 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:32.970904 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.971485 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.972762 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.972995 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.973108 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:32.976224 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:32.976225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:32.976226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:32.976235 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:32.978066 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:32.978602 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:32.978604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:32.978604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:32.978683 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:32.979110 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:32.979112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:32.979112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:32.979288 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:32.979380 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:32.979863 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 00:42:32.979970 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:42:32.980363 27125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:32.968385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:41969 (local address 127.20.105.67:39407)
0808 00:42:32.968592 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:32.968640 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:32.969806 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:32.970148 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:32.970159 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:32.970514 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:32.970786 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:32.970792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.971599 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.971600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:32.972652 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:32.972656 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:32.973127 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:32.973129 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:32.973291 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:32.973925 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:32.973936 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:32.976107 (+ 2171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:32.978169 (+ 2062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:32.978178 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:32.978181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:32.978488 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:32.978848 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:32.978849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:32.978850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:32.979004 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:32.979389 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:32.979407 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:32.979670 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 00:42:32.979791 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 00:42:32.979976 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
I20260808 00:42:32.980998 27123 heartbeater.cc:347] Connected to a master server at 127.20.105.126:41969
I20260808 00:42:32.981137 27123 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:32.981380 27123 heartbeater.cc:511] Master 127.20.105.126:41969 requested a full tablet report, sending...
I20260808 00:42:32.982005 26446 ts_manager.cc:194] Registered new tserver with Master: 4dccddc538c94caebd59c474e9a2bfc6 (127.20.105.67:35901)
I20260808 00:42:32.982462 26446 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:39407
I20260808 00:42:32.987385 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:42:32 dist-test-slave-kvfs krb5kdc[26384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:42:33 dist-test-slave-kvfs krb5kdc[26384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:33.017442 27132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:33.008808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.20.105.65:36371)
0808 00:42:33.009032 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:33.009039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:33.009053 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:42:33.009095 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:33.009098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:33.009117 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:33.009212 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:33.009859 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:33.010681 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:33.011450 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:33.011839 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:33.011898 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:33.014522 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:33.014544 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:33.014545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:33.014554 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:33.016290 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:33.016645 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:33.016648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:33.016649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:33.016717 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:33.016885 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:33.016885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:33.016886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:33.017045 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:33.017165 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:33.017236 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 00:42:33.017304 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":120,"threads_started":1}
Aug 08 00:42:33 dist-test-slave-kvfs krb5kdc[26384](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149752, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:33.026958 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:33.019375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51226 (local address 127.20.105.126:41969)
0808 00:42:33.019438 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:33.019441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:33.019450 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:42:33.019522 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:33.019524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:33.019536 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:33.019595 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:33.020235 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:33.020792 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:33.021548 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:33.021763 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:33.021825 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:33.024417 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:33.024419 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:33.024419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:33.024428 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:33.025782 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:33.026127 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:33.026128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:33.026128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:33.026185 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:33.026365 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:33.026366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:33.026366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:33.026550 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:33.026650 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:33.026797 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 00:42:33.026869 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 00:42:33.027302 26446 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51226
I20260808 00:42:33.027881 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 26543
I20260808 00:42:33.046943 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 26740
I20260808 00:42:33.065791 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 26932
I20260808 00:42:33.076679 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 26402
2026-08-08T00:42:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2269 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:33 dist-test-slave-kvfs krb5kdc[27138](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:33 dist-test-slave-kvfs krb5kdc[27138](info): set up 2 sockets
Aug 08 00:42:33 dist-test-slave-kvfs krb5kdc[27138](info): commencing operation
krb5kdc: starting...
W20260808 00:42:34.391671 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:34 dist-test-slave-kvfs krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:34.539364 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:46433
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:46433
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:34.639149 27157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:34.639482 27157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:34.639524 27157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:34.640851 27157 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:34.640937 27157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:34.640961 27157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:34.640985 27157 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:34.641010 27157 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:34.642756 27157 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:46433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:46433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:34.643110 27157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:34.643399 27157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:34.649148 27166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:34.649148 27163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:34.649199 27168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:34.649480 27157 server_base.cc:1034] running on GCE node
I20260808 00:42:34.649825 27157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:34.650179 27157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:34.651357 27157 hybrid_clock.cc:648] HybridClock initialized: now 1786149754651339 us; error 45 us; skew 500 ppm
Aug 08 00:42:34 dist-test-slave-kvfs krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149754, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:34.654697 27157 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:34.655162 27157 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:34.655220 27157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:34.655340 27157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:34.656749 27157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "210b36bd68de4de3a447c50c3a24402b"
format_stamp: "Formatted at 2026-08-08 00:42:34 on dist-test-slave-kvfs"
server_key: "8c7b25ee62b9b295dd4cc715e6c76d5a"
server_key_iv: "c334f4b89e4b6adabd252c84553d101f"
server_key_version: "encryptionkey@0"
I20260808 00:42:34.657159 27157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "210b36bd68de4de3a447c50c3a24402b"
format_stamp: "Formatted at 2026-08-08 00:42:34 on dist-test-slave-kvfs"
server_key: "8c7b25ee62b9b295dd4cc715e6c76d5a"
server_key_iv: "c334f4b89e4b6adabd252c84553d101f"
server_key_version: "encryptionkey@0"
I20260808 00:42:34.658789 27157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:42:34.659727 27180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:34.660005 27157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:34.660172 27157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "210b36bd68de4de3a447c50c3a24402b"
format_stamp: "Formatted at 2026-08-08 00:42:34 on dist-test-slave-kvfs"
server_key: "8c7b25ee62b9b295dd4cc715e6c76d5a"
server_key_iv: "c334f4b89e4b6adabd252c84553d101f"
server_key_version: "encryptionkey@0"
I20260808 00:42:34.660260 27157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:34.680166 27157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:34.682914 27157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:34.683135 27157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:34.688525 27157 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:46433
I20260808 00:42:34.688549 27255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:46433 every 8 connection(s)
I20260808 00:42:34.689007 27157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:34.690372 27257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:34.693589 27257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b: Bootstrap starting.
I20260808 00:42:34.694368 27257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:34.694680 27257 log.cc:824] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:34.695500 27257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b: No bootstrap required, opened a new log
I20260808 00:42:34.697396 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 27157
I20260808 00:42:34.697552 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:34.697453 27257 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210b36bd68de4de3a447c50c3a24402b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46433 } }
I20260808 00:42:34.697706 27257 raft_consensus.cc:374] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:34.697754 27257 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210b36bd68de4de3a447c50c3a24402b, State: Initialized, Role: FOLLOWER
I20260808 00:42:34.697815 20901 external_mini_cluster.cc:1428] Setting key a6510fc4489398bff766ed3fcced4770
I20260808 00:42:34.697947 27257 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210b36bd68de4de3a447c50c3a24402b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46433 } }
I20260808 00:42:34.698086 27257 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:34.698132 27257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:34.698221 27257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:34.699919 27257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210b36bd68de4de3a447c50c3a24402b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46433 } }
I20260808 00:42:34.700105 27257 leader_election.cc:302] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210b36bd68de4de3a447c50c3a24402b; no voters:
I20260808 00:42:34.700335 27257 leader_election.cc:288] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:34.700450 27264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:34.700642 27264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [term 1 LEADER]: Becoming Leader. State: Replica: 210b36bd68de4de3a447c50c3a24402b, State: Running, Role: LEADER
I20260808 00:42:34.700757 27257 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:34.700807 27264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210b36bd68de4de3a447c50c3a24402b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46433 } }
I20260808 00:42:34.701296 27264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210b36bd68de4de3a447c50c3a24402b. Latest consensus state: current_term: 1 leader_uuid: "210b36bd68de4de3a447c50c3a24402b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210b36bd68de4de3a447c50c3a24402b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46433 } } }
I20260808 00:42:34.701399 27264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:34.701308 27265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "210b36bd68de4de3a447c50c3a24402b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210b36bd68de4de3a447c50c3a24402b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46433 } } }
I20260808 00:42:34.701727 27265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:34.702443 27272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:34.703133 27272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:34.705492 27272 catalog_manager.cc:1347] Generated new cluster ID: 074e761b645d426494b848f2d68d314d
I20260808 00:42:34.705559 27272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:42:34 dist-test-slave-kvfs krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:34.711688 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:34.698578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42682 (local address 127.20.105.126:46433)
0808 00:42:34.698818 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:34.698830 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:34.698880 (+ 50us) server_negotiation.cc:407] Connection header received
0808 00:42:34.699055 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:34.699065 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:34.699177 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:34.699386 (+ 209us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:34.700153 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:34.701888 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:34.703094 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:34.703654 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:34.703784 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:34.707345 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:34.707351 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:34.707352 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:34.707368 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:34.709650 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:34.710184 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:34.710188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:34.710188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:34.710336 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:34.710589 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:34.710590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:34.710590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:34.710952 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:34.711128 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:34.711261 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 00:42:34.711418 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":130,"threads_started":1}
I20260808 00:42:34.715421 27272 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:34.716291 27272 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:34.720566 27272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b: Generated new TSK 0
I20260808 00:42:34.720788 27272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:34.755990 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46433
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:34.855165 27290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:34.855454 27290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:34.855496 27290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:34.856815 27290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:34.856885 27290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:34.856935 27290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:34.858649 27290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:34.858973 27290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:34.859265 27290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:34.865340 27302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:34.865399 27300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:34.865396 27299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:34.865777 27290 server_base.cc:1034] running on GCE node
I20260808 00:42:34.866014 27290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:34.866310 27290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:34.867465 27290 hybrid_clock.cc:648] HybridClock initialized: now 1786149754867457 us; error 39 us; skew 500 ppm
Aug 08 00:42:34 dist-test-slave-kvfs krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149754, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:34.870432 27290 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:34.870925 27290 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:34.871001 27290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:34.871171 27290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:34.872539 27290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "ba89cd5434584d8a9d11486eb394b89f"
format_stamp: "Formatted at 2026-08-08 00:42:34 on dist-test-slave-kvfs"
server_key: "ca1915f817fd38f5b6c5a832cba8fae8"
server_key_iv: "e45c4608b86dcaa1cb32ccf07b1524e4"
server_key_version: "encryptionkey@0"
I20260808 00:42:34.872972 27290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "ba89cd5434584d8a9d11486eb394b89f"
format_stamp: "Formatted at 2026-08-08 00:42:34 on dist-test-slave-kvfs"
server_key: "ca1915f817fd38f5b6c5a832cba8fae8"
server_key_iv: "e45c4608b86dcaa1cb32ccf07b1524e4"
server_key_version: "encryptionkey@0"
I20260808 00:42:34.874569 27290 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:34.875435 27311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:34.875639 27290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:34.875756 27290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "ba89cd5434584d8a9d11486eb394b89f"
format_stamp: "Formatted at 2026-08-08 00:42:34 on dist-test-slave-kvfs"
server_key: "ca1915f817fd38f5b6c5a832cba8fae8"
server_key_iv: "e45c4608b86dcaa1cb32ccf07b1524e4"
server_key_version: "encryptionkey@0"
I20260808 00:42:34.875838 27290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:34.888273 27290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:34.890710 27290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:34.890910 27290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:34.891204 27290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:34.891625 27290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:34.891682 27290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:34.891723 27290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:34.891764 27290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:34.899071 27290 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:33229
I20260808 00:42:34.899120 27469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:33229 every 8 connection(s)
I20260808 00:42:34.899456 27290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:34.904124 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 27290
I20260808 00:42:34.904274 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:34.904505 20901 external_mini_cluster.cc:1428] Setting key e0333fd23dd712df9cef8218e182d0c2
Aug 08 00:42:34 dist-test-slave-kvfs krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149754, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:34.911375 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:34.900460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:60415 (local address 127.20.105.126:46433)
0808 00:42:34.900536 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:34.900538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:34.900769 (+ 231us) server_negotiation.cc:407] Connection header received
0808 00:42:34.901480 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:34.901482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:34.901504 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:34.901556 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:34.902881 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:34.903468 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:34.904980 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:34.905212 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:34.905303 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:34.907824 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:34.907826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:34.907826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:34.907841 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:34.909661 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:34.910214 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:34.910214 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:34.910215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:34.910273 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:34.910570 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:34.910571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:34.910571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:34.910700 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:34.910777 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:34.911181 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 00:42:34.911280 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:42:34.911715 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:34.900443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:46433 (local address 127.20.105.65:60415)
0808 00:42:34.900611 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:34.900660 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:34.901348 (+ 688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:34.901652 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:34.901657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:34.902074 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:34.902764 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:34.902771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:34.903571 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:34.903573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:34.904876 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:34.904879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:34.905313 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:34.905314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:34.905460 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:34.906089 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:34.906099 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:34.907705 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:34.909756 (+ 2051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:34.909766 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:34.909769 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:34.910135 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:34.910337 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:34.910338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:34.910339 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:34.910508 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:34.910781 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:34.910789 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:34.910987 (+ 198us) client_negotiation.cc:769] Sending connection context
0808 00:42:34.911116 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 00:42:34.911348 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260808 00:42:34.912493 27472 heartbeater.cc:347] Connected to a master server at 127.20.105.126:46433
I20260808 00:42:34.912643 27472 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:34.912843 27472 heartbeater.cc:511] Master 127.20.105.126:46433 requested a full tablet report, sending...
I20260808 00:42:34.913568 27201 ts_manager.cc:194] Registered new tserver with Master: ba89cd5434584d8a9d11486eb394b89f (127.20.105.65:33229)
I20260808 00:42:34.914287 27201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:60415
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:34.939664 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46433
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:35.037922 27479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:35.038249 27479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:35.038292 27479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:35.039546 27479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:35.039629 27479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:35.039681 27479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:35.041452 27479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:35.041801 27479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:35.042111 27479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:35.048578 27485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:35.048581 27489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:35.048738 27479 server_base.cc:1034] running on GCE node
W20260808 00:42:35.048627 27486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:35.049219 27479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:35.049489 27479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:35.050647 27479 hybrid_clock.cc:648] HybridClock initialized: now 1786149755050626 us; error 42 us; skew 500 ppm
Aug 08 00:42:35 dist-test-slave-kvfs krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149755, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:35.053850 27479 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:35.054349 27479 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:35.054425 27479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:35.054602 27479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:35.055902 27479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "5fb90452702542c4988b814ded9caab0"
format_stamp: "Formatted at 2026-08-08 00:42:35 on dist-test-slave-kvfs"
server_key: "2df95d02ecf8b7f1e30d43d2daeb9fd0"
server_key_iv: "445b1106eb77f0b76f8ad8b4a9a941bf"
server_key_version: "encryptionkey@0"
I20260808 00:42:35.056519 27479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "5fb90452702542c4988b814ded9caab0"
format_stamp: "Formatted at 2026-08-08 00:42:35 on dist-test-slave-kvfs"
server_key: "2df95d02ecf8b7f1e30d43d2daeb9fd0"
server_key_iv: "445b1106eb77f0b76f8ad8b4a9a941bf"
server_key_version: "encryptionkey@0"
I20260808 00:42:35.058332 27479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:35.059188 27498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:35.059384 27479 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:35.059497 27479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "5fb90452702542c4988b814ded9caab0"
format_stamp: "Formatted at 2026-08-08 00:42:35 on dist-test-slave-kvfs"
server_key: "2df95d02ecf8b7f1e30d43d2daeb9fd0"
server_key_iv: "445b1106eb77f0b76f8ad8b4a9a941bf"
server_key_version: "encryptionkey@0"
I20260808 00:42:35.059584 27479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:35.074244 27479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:35.077062 27479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:35.077257 27479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:35.077561 27479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:35.077955 27479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:35.078008 27479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:35.078078 27479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:35.078110 27479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:35.085242 27479 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:42015
I20260808 00:42:35.085285 27655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:42015 every 8 connection(s)
I20260808 00:42:35.085625 27479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:35.087793 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 27479
I20260808 00:42:35.087895 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:35.088174 20901 external_mini_cluster.cc:1428] Setting key 07d37728c6d29ddbc92769f8f0c1b5fa
Aug 08 00:42:35 dist-test-slave-kvfs krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149755, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:35.097306 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.086506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:36689 (local address 127.20.105.126:46433)
0808 00:42:35.086588 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.086603 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.086827 (+ 224us) server_negotiation.cc:407] Connection header received
0808 00:42:35.087550 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.087551 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.087616 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.087698 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:35.089047 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.089611 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.090764 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.090982 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.091062 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.094006 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:35.094008 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:35.094008 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:35.094016 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:35.095590 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:35.096161 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:35.096162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:35.096162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:35.096224 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:35.096543 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:35.096543 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:35.096544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:35.096668 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:35.096722 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.097136 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.097222 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:42:35.097662 27662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.086466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:46433 (local address 127.20.105.66:36689)
0808 00:42:35.086678 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.086728 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:35.087422 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:35.087795 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:35.087800 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:35.088318 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:35.088938 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.088946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.089716 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.089718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.090654 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.090656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.091077 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.091078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.091242 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.091871 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:35.091881 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:35.093903 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:35.095674 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:35.095682 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:35.095686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:35.096075 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:35.096291 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:35.096292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:35.096293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:35.096463 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:35.096740 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:35.096749 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:35.096972 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 00:42:35.097111 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 00:42:35.097334 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":107,"threads_started":1}
I20260808 00:42:35.098330 27657 heartbeater.cc:347] Connected to a master server at 127.20.105.126:46433
I20260808 00:42:35.098469 27657 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:35.098685 27657 heartbeater.cc:511] Master 127.20.105.126:46433 requested a full tablet report, sending...
I20260808 00:42:35.099136 27201 ts_manager.cc:194] Registered new tserver with Master: 5fb90452702542c4988b814ded9caab0 (127.20.105.66:42015)
I20260808 00:42:35.099581 27201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:36689
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:35.122825 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46433
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:35.222138 27665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:35.222424 27665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:35.222452 27665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:35.223706 27665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:35.223747 27665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:35.223775 27665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:35.225713 27665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:35.225991 27665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:35.226243 27665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:35.232506 27678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:35.232651 27675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:35.232810 27676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:35.233184 27665 server_base.cc:1034] running on GCE node
I20260808 00:42:35.233522 27665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:35.233872 27665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:35.235080 27665 hybrid_clock.cc:648] HybridClock initialized: now 1786149755235044 us; error 63 us; skew 500 ppm
Aug 08 00:42:35 dist-test-slave-kvfs krb5kdc[27138](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149755, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:35.238875 27665 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:35.239466 27665 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:35.239539 27665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:35.239682 27665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:35.241127 27665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "2f2c1914237d4398bacc8e0550b56e33"
format_stamp: "Formatted at 2026-08-08 00:42:35 on dist-test-slave-kvfs"
server_key: "7ce7dbf575345bf751a91c1f21bdfea6"
server_key_iv: "3299836ec6d118fcc9aa4ea880bd7814"
server_key_version: "encryptionkey@0"
I20260808 00:42:35.241662 27665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "2f2c1914237d4398bacc8e0550b56e33"
format_stamp: "Formatted at 2026-08-08 00:42:35 on dist-test-slave-kvfs"
server_key: "7ce7dbf575345bf751a91c1f21bdfea6"
server_key_iv: "3299836ec6d118fcc9aa4ea880bd7814"
server_key_version: "encryptionkey@0"
I20260808 00:42:35.243623 27665 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 00:42:35.253376 27690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:35.256414 27665 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.000s sys 0.001s
I20260808 00:42:35.256635 27665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "2f2c1914237d4398bacc8e0550b56e33"
format_stamp: "Formatted at 2026-08-08 00:42:35 on dist-test-slave-kvfs"
server_key: "7ce7dbf575345bf751a91c1f21bdfea6"
server_key_iv: "3299836ec6d118fcc9aa4ea880bd7814"
server_key_version: "encryptionkey@0"
I20260808 00:42:35.256798 27665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:35.278258 27665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:35.282032 27665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:35.282353 27665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:35.282776 27665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:35.283252 27665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:35.283325 27665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:35.283380 27665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:35.283408 27665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:35.290876 27665 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:43761
I20260808 00:42:35.290922 27858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:43761 every 8 connection(s)
I20260808 00:42:35.291303 27665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:35.293004 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 27665
I20260808 00:42:35.293095 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:35.293334 20901 external_mini_cluster.cc:1428] Setting key 56cdf1df5f1e71dd7b8336350b97d48c
Aug 08 00:42:35 dist-test-slave-kvfs krb5kdc[27138](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149755, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:35.302989 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.292346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:42359 (local address 127.20.105.126:46433)
0808 00:42:35.292407 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.292409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.292714 (+ 305us) server_negotiation.cc:407] Connection header received
0808 00:42:35.293508 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.293509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.293526 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.293595 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:35.295078 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.295650 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.296875 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.297118 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.297188 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.299689 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:35.299691 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:35.299691 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:35.299700 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:35.301265 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:35.301719 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:35.301720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:35.301721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:35.301850 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:35.302138 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:35.302139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:35.302140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:35.302286 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:35.302352 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.302737 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.302831 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:42:35.303169 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.292351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:46433 (local address 127.20.105.67:42359)
0808 00:42:35.292555 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.292606 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:35.293319 (+ 713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:35.293730 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:35.293746 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:35.294250 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:35.294975 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.294982 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.295742 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.295743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.296776 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.296778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.297202 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.297203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.297326 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.297933 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:35.297943 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:35.299582 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:35.301352 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:35.301359 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:35.301361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:35.301623 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:35.301923 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:35.301925 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:35.301925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:35.302066 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:35.302367 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:35.302374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:35.302544 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 00:42:35.302659 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 00:42:35.302826 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260808 00:42:35.303893 27859 heartbeater.cc:347] Connected to a master server at 127.20.105.126:46433
I20260808 00:42:35.304077 27859 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:35.304297 27859 heartbeater.cc:511] Master 127.20.105.126:46433 requested a full tablet report, sending...
I20260808 00:42:35.304996 27201 ts_manager.cc:194] Registered new tserver with Master: 2f2c1914237d4398bacc8e0550b56e33 (127.20.105.67:43761)
I20260808 00:42:35.305541 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:42:35.305640 27201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:42359
I20260808 00:42:35.314864 27263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.307612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42688 (local address 127.20.105.126:46433)
0808 00:42:35.307681 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.307683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.307842 (+ 159us) server_negotiation.cc:407] Connection header received
0808 00:42:35.307887 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.307891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.307904 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.307970 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:35.308682 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.309260 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.310112 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.310333 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.310403 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.312010 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:35.312012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:35.312013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:35.312071 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:35.313632 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:35.313997 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:35.313998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:35.313999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:35.314065 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:35.314250 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:35.314251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:35.314251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:35.314458 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:35.314555 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.314695 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.314773 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:42:35.316964 27201 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42688:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:35.319044 27201 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:35.327842 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.322161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:43761 (local address 127.0.0.1:33960)
0808 00:42:35.322851 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.322863 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:35.322946 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:35.323271 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:35.323272 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:35.323273 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:35.323352 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.323354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.324706 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.324707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.326049 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.326053 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.327205 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.327207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.327406 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.327426 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 00:42:35.327645 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 00:42:35.327687 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":8,"thread_start_us":88,"threads_started":1}
I20260808 00:42:35.327828 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.322327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52604 (local address 127.20.105.66:42015)
0808 00:42:35.322565 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.322573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.322586 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:42:35.322678 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.322681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.322701 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.322785 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:35.323139 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.324336 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.326442 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.326974 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.327186 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.327274 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.327624 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.327674 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":130,"threads_started":1}
I20260808 00:42:35.327908 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.322672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33960 (local address 127.20.105.67:43761)
0808 00:42:35.322943 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.322948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.322959 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:35.323037 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.323040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.323059 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.323151 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:35.323448 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.324598 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.326177 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.326996 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.327082 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.327161 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.327750 (+ 589us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.327811 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":99,"threads_started":1}
I20260808 00:42:35.327986 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.322530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52408 (local address 127.20.105.65:33229)
0808 00:42:35.322789 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.322794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.323185 (+ 391us) server_negotiation.cc:407] Connection header received
0808 00:42:35.323243 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.323246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.323264 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.323352 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:35.323602 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.324478 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.326423 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.327169 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.327236 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.327292 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.327793 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.327842 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":113,"threads_started":1}
I20260808 00:42:35.328223 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.322161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:42015 (local address 127.0.0.1:52604)
0808 00:42:35.322383 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.322436 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:35.322560 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:35.322914 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:35.322918 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:35.322921 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:35.323046 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.323049 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.324460 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.324461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.326327 (+ 1866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.326329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.327304 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.327305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.327404 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.327432 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 00:42:35.327503 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:42:35.328143 (+ 640us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":753,"thread_start_us":115,"threads_started":1}
I20260808 00:42:35.328812 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.322174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:33229 (local address 127.0.0.1:52408)
0808 00:42:35.323044 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.323054 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:35.323113 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:35.323471 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:35.323471 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:35.323473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:35.323515 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.323516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.324589 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.324590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.326321 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.326326 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.327328 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.327329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.327397 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.327426 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 00:42:35.327504 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 00:42:35.327675 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":122,"thread_start_us":131,"threads_started":1}
I20260808 00:42:35.328992 27754 tablet_service.cc:1467] Processing CreateTablet for tablet 433f587586294922bc50ba3a3c960a74 (DEFAULT_TABLE table=test-table [id=233b5fac885340d0999a7ffec50b65e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:35.329299 27558 tablet_service.cc:1467] Processing CreateTablet for tablet 433f587586294922bc50ba3a3c960a74 (DEFAULT_TABLE table=test-table [id=233b5fac885340d0999a7ffec50b65e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:35.329641 27754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 433f587586294922bc50ba3a3c960a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:35.329687 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 433f587586294922bc50ba3a3c960a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:35.329905 27378 tablet_service.cc:1467] Processing CreateTablet for tablet 433f587586294922bc50ba3a3c960a74 (DEFAULT_TABLE table=test-table [id=233b5fac885340d0999a7ffec50b65e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:35.330294 27378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 433f587586294922bc50ba3a3c960a74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:35.333997 27889 tablet_bootstrap.cc:492] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: Bootstrap starting.
I20260808 00:42:35.334501 27890 tablet_bootstrap.cc:492] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Bootstrap starting.
I20260808 00:42:35.334594 27891 tablet_bootstrap.cc:492] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f: Bootstrap starting.
I20260808 00:42:35.335194 27889 tablet_bootstrap.cc:654] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:35.335340 27891 tablet_bootstrap.cc:654] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:35.335336 27890 tablet_bootstrap.cc:654] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:35.335850 27889 log.cc:824] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:35.336127 27890 log.cc:824] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:35.336127 27891 log.cc:824] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:35.336915 27889 tablet_bootstrap.cc:492] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: No bootstrap required, opened a new log
I20260808 00:42:35.336982 27891 tablet_bootstrap.cc:492] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f: No bootstrap required, opened a new log
I20260808 00:42:35.336997 27890 tablet_bootstrap.cc:492] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: No bootstrap required, opened a new log
I20260808 00:42:35.337021 27889 ts_tablet_manager.cc:1397] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:42:35.337060 27890 ts_tablet_manager.cc:1397] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:42:35.337060 27891 ts_tablet_manager.cc:1397] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:42:35.338948 27889 raft_consensus.cc:348] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.339141 27889 raft_consensus.cc:374] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:35.339191 27889 raft_consensus.cc:729] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb90452702542c4988b814ded9caab0, State: Initialized, Role: FOLLOWER
I20260808 00:42:35.339330 27889 consensus_queue.cc:260] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.339346 27890 raft_consensus.cc:348] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.339346 27891 raft_consensus.cc:348] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.339522 27891 raft_consensus.cc:374] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:35.339522 27890 raft_consensus.cc:374] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:35.339558 27890 raft_consensus.cc:729] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2c1914237d4398bacc8e0550b56e33, State: Initialized, Role: FOLLOWER
I20260808 00:42:35.339558 27891 raft_consensus.cc:729] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba89cd5434584d8a9d11486eb394b89f, State: Initialized, Role: FOLLOWER
I20260808 00:42:35.339704 27889 ts_tablet_manager.cc:1428] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:42:35.339720 27890 consensus_queue.cc:260] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.339767 27657 heartbeater.cc:503] Master 127.20.105.126:46433 was elected leader, sending a full tablet report...
I20260808 00:42:35.339720 27891 consensus_queue.cc:260] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.340077 27891 ts_tablet_manager.cc:1428] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:42:35.340077 27890 ts_tablet_manager.cc:1428] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:42:35.340101 27472 heartbeater.cc:503] Master 127.20.105.126:46433 was elected leader, sending a full tablet report...
I20260808 00:42:35.340212 27859 heartbeater.cc:503] Master 127.20.105.126:46433 was elected leader, sending a full tablet report...
W20260808 00:42:35.400431 27473 tablet.cc:2369] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:42:35.542076 27861 tablet.cc:2369] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:42:35.586432 27661 tablet.cc:2369] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:35.728111 27900 raft_consensus.cc:482] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:35.728361 27900 raft_consensus.cc:504] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.729072 27900 leader_election.cc:288] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fb90452702542c4988b814ded9caab0 (127.20.105.66:42015), ba89cd5434584d8a9d11486eb394b89f (127.20.105.65:33229)
I20260808 00:42:35.733412 27885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.729685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:49003 (local address 127.20.105.65:33229)
0808 00:42:35.729771 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.729773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.729784 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:35.729826 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.729828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.729840 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.729927 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:35.730138 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.730741 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.732519 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.732955 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.733055 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.733070 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.733262 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.733319 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:42:35.733410 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.729399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:33229 (local address 127.20.105.67:49003)
0808 00:42:35.729489 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.729505 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:35.729568 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:35.729947 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:35.729949 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:35.729951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:35.730055 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.730057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.730846 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.730848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.732342 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.732347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.733054 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.733055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.733170 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.733174 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:42:35.733251 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 00:42:35.733307 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":29}
I20260808 00:42:35.733476 27884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.729523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:45587 (local address 127.20.105.66:42015)
0808 00:42:35.729579 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.729581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.729745 (+ 164us) server_negotiation.cc:407] Connection header received
0808 00:42:35.729858 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.729860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.729869 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.729947 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:35.730260 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.730756 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.732470 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.733039 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.733141 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.733153 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.733340 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.733397 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:42:35.733551 27903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.729399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:42015 (local address 127.20.105.67:45587)
0808 00:42:35.729676 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.729686 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:35.729775 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:35.730126 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:35.730127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:35.730128 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:35.730176 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.730177 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.730883 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.730883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.732342 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:35.732347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.733162 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.733164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:35.733256 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.733260 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:42:35.733330 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:42:35.733379 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":58,"thread_start_us":173,"threads_started":1}
I20260808 00:42:35.733930 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "433f587586294922bc50ba3a3c960a74" candidate_uuid: "2f2c1914237d4398bacc8e0550b56e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba89cd5434584d8a9d11486eb394b89f" is_pre_election: true
I20260808 00:42:35.733925 27590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "433f587586294922bc50ba3a3c960a74" candidate_uuid: "2f2c1914237d4398bacc8e0550b56e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fb90452702542c4988b814ded9caab0" is_pre_election: true
I20260808 00:42:35.734158 27402 raft_consensus.cc:2454] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f2c1914237d4398bacc8e0550b56e33 in term 0.
I20260808 00:42:35.734158 27590 raft_consensus.cc:2454] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f2c1914237d4398bacc8e0550b56e33 in term 0.
I20260808 00:42:35.734457 27693 leader_election.cc:302] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2c1914237d4398bacc8e0550b56e33, ba89cd5434584d8a9d11486eb394b89f; no voters:
I20260808 00:42:35.734656 27900 raft_consensus.cc:2781] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:35.734730 27900 raft_consensus.cc:482] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:35.734750 27900 raft_consensus.cc:3037] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:35.739765 27900 raft_consensus.cc:504] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.740149 27900 leader_election.cc:288] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [CANDIDATE]: Term 1 election: Requested vote from peers 5fb90452702542c4988b814ded9caab0 (127.20.105.66:42015), ba89cd5434584d8a9d11486eb394b89f (127.20.105.65:33229)
I20260808 00:42:35.740491 27590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "433f587586294922bc50ba3a3c960a74" candidate_uuid: "2f2c1914237d4398bacc8e0550b56e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fb90452702542c4988b814ded9caab0"
I20260808 00:42:35.740494 27402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "433f587586294922bc50ba3a3c960a74" candidate_uuid: "2f2c1914237d4398bacc8e0550b56e33" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba89cd5434584d8a9d11486eb394b89f"
I20260808 00:42:35.740651 27590 raft_consensus.cc:3037] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:35.740650 27402 raft_consensus.cc:3037] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:35.741755 27590 raft_consensus.cc:2454] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f2c1914237d4398bacc8e0550b56e33 in term 1.
I20260808 00:42:35.742071 27402 raft_consensus.cc:2454] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f2c1914237d4398bacc8e0550b56e33 in term 1.
I20260808 00:42:35.742074 27694 leader_election.cc:302] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f2c1914237d4398bacc8e0550b56e33, 5fb90452702542c4988b814ded9caab0; no voters:
I20260808 00:42:35.742239 27900 raft_consensus.cc:2781] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:35.742476 27900 raft_consensus.cc:686] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 1 LEADER]: Becoming Leader. State: Replica: 2f2c1914237d4398bacc8e0550b56e33, State: Running, Role: LEADER
I20260808 00:42:35.742643 27900 consensus_queue.cc:237] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } }
I20260808 00:42:35.743969 27201 catalog_manager.cc:5495] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f2c1914237d4398bacc8e0550b56e33 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "2f2c1914237d4398bacc8e0550b56e33" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f2c1914237d4398bacc8e0550b56e33" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:35.776142 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.773056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33972 (local address 127.20.105.67:43761)
0808 00:42:35.773155 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:35.773158 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:35.773193 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:42:35.773253 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:35.773254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:35.773278 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:35.773365 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:35.773553 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.774171 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.775109 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:35.775330 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:35.775407 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:35.775624 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:35.775778 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:35.775892 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:35.775910 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 00:42:35.775941 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 00:42:35.778808 27402 raft_consensus.cc:1261] T 433f587586294922bc50ba3a3c960a74 P ba89cd5434584d8a9d11486eb394b89f [term 1 FOLLOWER]: Refusing update from remote peer 2f2c1914237d4398bacc8e0550b56e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:35.778990 27590 raft_consensus.cc:1261] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 1 FOLLOWER]: Refusing update from remote peer 2f2c1914237d4398bacc8e0550b56e33: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:35.779312 27900 consensus_queue.cc:1035] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba89cd5434584d8a9d11486eb394b89f" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 33229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:35.779592 27904 consensus_queue.cc:1035] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fb90452702542c4988b814ded9caab0" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:35.783804 27910 mvcc.cc:205] Tried to move back new op lower bound from 7316069399665876992 to 7316069399522676736. Current Snapshot: MvccSnapshot[applied={T|T < 7316069399665876992}]
I20260808 00:42:35.786521 27201 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:35.786674 27201 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42688:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:35.788075 27201 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210b36bd68de4de3a447c50c3a24402b: Sending DeleteTablet for 3 replicas of tablet 433f587586294922bc50ba3a3c960a74
I20260808 00:42:35.788532 27754 tablet_service.cc:1514] Processing DeleteTablet for tablet 433f587586294922bc50ba3a3c960a74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:35 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:33960
I20260808 00:42:35.788604 27378 tablet_service.cc:1514] Processing DeleteTablet for tablet 433f587586294922bc50ba3a3c960a74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:35 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:52408
I20260808 00:42:35.788967 27558 tablet_service.cc:1514] Processing DeleteTablet for tablet 433f587586294922bc50ba3a3c960a74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:35 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:52604
I20260808 00:42:35.789148 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 27290
I20260808 00:42:35.789043 27926 tablet_replica.cc:330] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: stopping tablet replica
I20260808 00:42:35.790557 27926 raft_consensus.cc:2229] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:35.790835 27926 raft_consensus.cc:2258] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:35.793416 27926 ts_tablet_manager.cc:1905] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:35.794862 27928 tablet_replica.cc:330] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: stopping tablet replica
I20260808 00:42:35.795066 27928 raft_consensus.cc:2229] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:35.795274 27928 raft_consensus.cc:2258] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:35.797794 27928 ts_tablet_manager.cc:1905] T 433f587586294922bc50ba3a3c960a74 P 5fb90452702542c4988b814ded9caab0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 00:42:35.810662 27183 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:33229 (error 108)
I20260808 00:42:35.811484 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.811300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:33229 (local address 127.0.0.1:52422)
0808 00:42:35.811376 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.811429 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:33229: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 00:42:35.811589 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 27479
I20260808 00:42:35.823060 27926 ts_tablet_manager.cc:1918] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:35.823251 27926 log.cc:1196] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/433f587586294922bc50ba3a3c960a74
I20260808 00:42:35.823657 27926 ts_tablet_manager.cc:1939] T 433f587586294922bc50ba3a3c960a74 P 2f2c1914237d4398bacc8e0550b56e33: Deleting consensus metadata
W20260808 00:42:35.824317 27183 catalog_manager.cc:4584] TS ba89cd5434584d8a9d11486eb394b89f (127.20.105.65:33229): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 433f587586294922bc50ba3a3c960a74: Network error: Client connection negotiation failed: client connection to 127.20.105.65:33229: connect: Connection refused (error 111)
I20260808 00:42:35.835418 27181 catalog_manager.cc:4841] TS 2f2c1914237d4398bacc8e0550b56e33 (127.20.105.67:43761): tablet 433f587586294922bc50ba3a3c960a74 (table test-table [id=233b5fac885340d0999a7ffec50b65e6]) successfully deleted
I20260808 00:42:35.836831 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 27665
I20260808 00:42:35.836874 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:35.836665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:42015 (local address 127.0.0.1:52612)
0808 00:42:35.836766 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:35.836812 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.66:42015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 00:42:35.856873 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 27157
2026-08-08T00:42:35Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2777 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T00:42:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:35Z Disabled control of system clock
I20260808 00:42:35.901626 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40435
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:42869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:40435
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:42:36.014840 27936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:36.015173 27936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:36.015200 27936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:36.016499 27936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:36.016574 27936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:36.016601 27936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:36.016615 27936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:36.016628 27936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:36.018409 27936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:42869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:40435
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:36.018720 27936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:36.019124 27936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:36.027611 27943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:36.027848 27944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:36.027971 27946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:36.028239 27936 server_base.cc:1034] running on GCE node
I20260808 00:42:36.028625 27936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:36.029037 27936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:36.030234 27936 hybrid_clock.cc:648] HybridClock initialized: now 1786149756030208 us; error 48 us; skew 500 ppm
I20260808 00:42:36.031877 27936 webserver.cc:460] Webserver started at http://127.20.105.126:42287/ using document root <none> and password file <none>
I20260808 00:42:36.032227 27936 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:36.032310 27936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:36.032490 27936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:36.033860 27936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "288bdf088507c953c6fdf19753d5cd5f"
server_key_iv: "2c26f2af3ccc15563f579637f2b5b4ef"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.034320 27936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "288bdf088507c953c6fdf19753d5cd5f"
server_key_iv: "2c26f2af3ccc15563f579637f2b5b4ef"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.036002 27936 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:42:36.037315 27957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.037526 27936 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:36.037647 27936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "288bdf088507c953c6fdf19753d5cd5f"
server_key_iv: "2c26f2af3ccc15563f579637f2b5b4ef"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.037750 27936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:36.041745 27936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:36.042093 27936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:36.042249 27936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:36.047279 27936 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:40435
I20260808 00:42:36.047318 28044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:40435 every 8 connection(s)
I20260808 00:42:36.047777 27936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:36.047987 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 27936
I20260808 00:42:36.048149 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:36.048396 20901 external_mini_cluster.cc:1428] Setting key 02a1f522af2de379ecd7dbbd79ffe775
I20260808 00:42:36.048833 28045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:36.051563 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.049080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45212 (local address 127.20.105.126:40435)
0808 00:42:36.049374 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.049387 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.049417 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:42:36.049980 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.049988 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.050036 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.050213 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.051074 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.051078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.051079 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.051083 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.051134 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.051145 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.051225 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.051300 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.051371 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":123,"threads_started":1}
I20260808 00:42:36.053277 28045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296: Bootstrap starting.
I20260808 00:42:36.054136 28045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:36.054486 28045 log.cc:824] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:36.055385 28045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296: No bootstrap required, opened a new log
I20260808 00:42:36.057210 28045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40435 } }
I20260808 00:42:36.057474 28045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:36.057530 28045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e31bf3b5d3a4bc78fe83574b6d7e296, State: Initialized, Role: FOLLOWER
I20260808 00:42:36.057682 28045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40435 } }
I20260808 00:42:36.057778 28045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:36.057821 28045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:36.057874 28045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:36.123514 28045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40435 } }
I20260808 00:42:36.123842 28045 leader_election.cc:302] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e31bf3b5d3a4bc78fe83574b6d7e296; no voters:
I20260808 00:42:36.124455 28045 leader_election.cc:288] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:36.124505 28050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:36.124713 28050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [term 1 LEADER]: Becoming Leader. State: Replica: 2e31bf3b5d3a4bc78fe83574b6d7e296, State: Running, Role: LEADER
I20260808 00:42:36.124853 28050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40435 } }
I20260808 00:42:36.124924 28045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:36.125409 28051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40435 } } }
I20260808 00:42:36.125519 28051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:36.125847 28053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e31bf3b5d3a4bc78fe83574b6d7e296. Latest consensus state: current_term: 1 leader_uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e31bf3b5d3a4bc78fe83574b6d7e296" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40435 } } }
I20260808 00:42:36.125978 28053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:36.126346 28056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:36.127307 28056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:36.131081 28056 catalog_manager.cc:1347] Generated new cluster ID: a69d96c4b53544bb911ea580a0575fe8
I20260808 00:42:36.131155 28056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:36.148280 28056 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:36.148561 28056 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:36.156275 28056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296: Generated new TSK 0
I20260808 00:42:36.156567 28056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:42:36.169548 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40435
--builtin_ntp_servers=127.20.105.84:42869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:42:36.286343 28077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:36.286648 28077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:36.286679 28077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:36.288305 28077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:36.288362 28077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:36.288403 28077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:36.290108 28077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:42869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:36.290445 28077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:36.290866 28077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:36.297391 28088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:36.297405 28084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:36.297689 28077 server_base.cc:1034] running on GCE node
W20260808 00:42:36.297451 28083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:36.298185 28077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:36.298660 28077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:36.299870 28077 hybrid_clock.cc:648] HybridClock initialized: now 1786149756299838 us; error 37 us; skew 500 ppm
I20260808 00:42:36.301576 28077 webserver.cc:460] Webserver started at http://127.20.105.65:39495/ using document root <none> and password file <none>
I20260808 00:42:36.301889 28077 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:36.301966 28077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:36.302156 28077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:36.303521 28077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "f622aa2b443144758b58044f5a93e08b"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "00fa9e205f74db802b2f82679ad592a6"
server_key_iv: "68c278dea313a9c058bbe388a104539c"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.303965 28077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "f622aa2b443144758b58044f5a93e08b"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "00fa9e205f74db802b2f82679ad592a6"
server_key_iv: "68c278dea313a9c058bbe388a104539c"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.305673 28077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:36.306802 28094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.307032 28077 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:36.307155 28077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "f622aa2b443144758b58044f5a93e08b"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "00fa9e205f74db802b2f82679ad592a6"
server_key_iv: "68c278dea313a9c058bbe388a104539c"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.307243 28077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:36.311661 28077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:36.312063 28077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:36.312219 28077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:36.312454 28077 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:36.312832 28077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:36.312886 28077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.312929 28077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:36.312959 28077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.320392 28077 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:46057
I20260808 00:42:36.320502 28259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:46057 every 8 connection(s)
I20260808 00:42:36.320927 28077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:36.325049 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.321881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:57491 (local address 127.20.105.126:40435)
0808 00:42:36.321998 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.322001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.322260 (+ 259us) server_negotiation.cc:407] Connection header received
0808 00:42:36.322937 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.322938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.322947 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.323006 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.324685 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.324687 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.324688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.324693 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.324710 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.324715 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.324820 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.324930 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.324947 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 00:42:36.325256 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.321839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40435 (local address 127.20.105.65:57491)
0808 00:42:36.322114 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.322157 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:36.322787 (+ 630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:36.323090 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:36.323094 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:36.323104 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:36.324479 (+ 1375us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:42:36.324480 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:36.324527 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:42:36.324566 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:36.324834 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:36.324842 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:36.324846 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:42:36.324926 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 00:42:36.324992 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":157,"threads_started":1}
I20260808 00:42:36.326124 28261 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40435
I20260808 00:42:36.326282 28261 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:36.326494 28261 heartbeater.cc:511] Master 127.20.105.126:40435 requested a full tablet report, sending...
I20260808 00:42:36.327282 27983 ts_manager.cc:194] Registered new tserver with Master: f622aa2b443144758b58044f5a93e08b (127.20.105.65:46057)
I20260808 00:42:36.329020 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 28077
I20260808 00:42:36.329144 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:36.329363 20901 external_mini_cluster.cc:1428] Setting key 2ad0b40a755ef1aa0105a84db0ffb88c
I20260808 00:42:36.330790 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40435
--builtin_ntp_servers=127.20.105.84:42869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:42:36.438823 28264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:36.439122 28264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:36.439152 28264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:36.440759 28264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:36.440824 28264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:36.440876 28264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:36.443302 28264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:42869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:36.443632 28264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:36.444224 28264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:42:36.451215 28264 server_base.cc:1034] running on GCE node
W20260808 00:42:36.451179 28276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:36.451215 28273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:36.451179 28272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:36.451829 28264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:36.452252 28264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:36.453450 28264 hybrid_clock.cc:648] HybridClock initialized: now 1786149756453422 us; error 53 us; skew 500 ppm
I20260808 00:42:36.455205 28264 webserver.cc:460] Webserver started at http://127.20.105.66:37071/ using document root <none> and password file <none>
I20260808 00:42:36.455574 28264 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:36.455654 28264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:36.455816 28264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:36.457259 28264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "bd0598086e444203b0bf7562244e08ee"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "6e91290407df7e046ced92c7a7ce43b3"
server_key_iv: "6b2644af548367b01958c01cfb739d69"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.458072 28264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "bd0598086e444203b0bf7562244e08ee"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "6e91290407df7e046ced92c7a7ce43b3"
server_key_iv: "6b2644af548367b01958c01cfb739d69"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.459921 28264 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:42:36.460934 28284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.461159 28264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:36.461282 28264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "bd0598086e444203b0bf7562244e08ee"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "6e91290407df7e046ced92c7a7ce43b3"
server_key_iv: "6b2644af548367b01958c01cfb739d69"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.461370 28264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:36.465945 28264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:36.466311 28264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:36.466496 28264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:36.466760 28264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:36.467196 28264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:36.467254 28264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.467303 28264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:36.467335 28264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.475390 28264 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:39833
I20260808 00:42:36.475431 28438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:39833 every 8 connection(s)
I20260808 00:42:36.475896 28264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:36.479077 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 28264
I20260808 00:42:36.479228 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:36.479501 20901 external_mini_cluster.cc:1428] Setting key 44bb032e2df5542e46c7b8ed8de46999
I20260808 00:42:36.481011 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.477034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:56163 (local address 127.20.105.126:40435)
0808 00:42:36.477197 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.477200 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.477442 (+ 242us) server_negotiation.cc:407] Connection header received
0808 00:42:36.478393 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.478395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.478404 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.478466 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.480293 (+ 1827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.480295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.480296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.480302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.480326 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.480331 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.480711 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.480877 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.480902 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 00:42:36.481280 28446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.476955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40435 (local address 127.20.105.66:56163)
0808 00:42:36.477255 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.477312 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:36.478027 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:36.478569 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:36.478577 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:36.478593 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:36.480092 (+ 1499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:42:36.480093 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:36.480145 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:42:36.480184 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:36.480750 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:36.480759 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:36.480764 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:36.480876 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 00:42:36.480973 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":109,"threads_started":1}
I20260808 00:42:36.481367 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40435
--builtin_ntp_servers=127.20.105.84:42869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 00:42:36.481958 28444 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40435
I20260808 00:42:36.482101 28444 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:36.482299 28444 heartbeater.cc:511] Master 127.20.105.126:40435 requested a full tablet report, sending...
I20260808 00:42:36.482800 27983 ts_manager.cc:194] Registered new tserver with Master: bd0598086e444203b0bf7562244e08ee (127.20.105.66:39833)
W20260808 00:42:36.588258 28447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:36.588533 28447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:36.588559 28447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:36.590119 28447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:36.590178 28447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:36.590210 28447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:36.591888 28447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:42869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40435
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:36.592253 28447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:36.592582 28447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:36.598624 28458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:36.598625 28455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:36.598624 28454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:36.599027 28447 server_base.cc:1034] running on GCE node
I20260808 00:42:36.599344 28447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:36.599690 28447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:36.600874 28447 hybrid_clock.cc:648] HybridClock initialized: now 1786149756600874 us; error 51 us; skew 500 ppm
I20260808 00:42:36.602727 28447 webserver.cc:460] Webserver started at http://127.20.105.67:42173/ using document root <none> and password file <none>
I20260808 00:42:36.603029 28447 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:36.603106 28447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:36.603287 28447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:36.604789 28447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "37be15b2c1174b94aa3645054baf074a"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "7bd7329c056108d6243cf1dcd12d3a7b"
server_key_iv: "87871ed638382f19b76287e5a4f252e2"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.605280 28447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "37be15b2c1174b94aa3645054baf074a"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "7bd7329c056108d6243cf1dcd12d3a7b"
server_key_iv: "87871ed638382f19b76287e5a4f252e2"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.607025 28447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:42:36.608096 28467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.608311 28447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:36.608420 28447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "37be15b2c1174b94aa3645054baf074a"
format_stamp: "Formatted at 2026-08-08 00:42:36 on dist-test-slave-kvfs"
server_key: "7bd7329c056108d6243cf1dcd12d3a7b"
server_key_iv: "87871ed638382f19b76287e5a4f252e2"
server_key_version: "encryptionkey@0"
I20260808 00:42:36.608498 28447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:36.613049 28447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:36.613435 28447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:36.613600 28447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:36.613853 28447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:36.614243 28447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:36.614313 28447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.614372 28447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:36.614403 28447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:36.621872 28447 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:46147
I20260808 00:42:36.621992 28634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:46147 every 8 connection(s)
I20260808 00:42:36.622399 28447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:36.626770 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.623404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:34731 (local address 127.20.105.126:40435)
0808 00:42:36.623479 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.623493 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.623764 (+ 271us) server_negotiation.cc:407] Connection header received
0808 00:42:36.624668 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.624670 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.624679 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.624775 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.626404 (+ 1629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.626405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.626406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.626411 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.626435 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.626441 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.626533 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.626658 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.626677 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:42:36.627002 28637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.623379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40435 (local address 127.20.105.67:34731)
0808 00:42:36.623619 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.623661 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:36.624540 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:36.624873 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:36.624877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:36.624887 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:36.626194 (+ 1307us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:42:36.626194 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:36.626232 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:42:36.626267 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:36.626533 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:36.626558 (+ 25us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:36.626568 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 00:42:36.626656 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:42:36.626737 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":112,"threads_started":1}
I20260808 00:42:36.627702 28635 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40435
I20260808 00:42:36.627844 28635 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:36.628098 28635 heartbeater.cc:511] Master 127.20.105.126:40435 requested a full tablet report, sending...
I20260808 00:42:36.628682 27983 ts_manager.cc:194] Registered new tserver with Master: 37be15b2c1174b94aa3645054baf074a (127.20.105.67:46147)
I20260808 00:42:36.630122 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 28447
I20260808 00:42:36.630225 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:36.630478 20901 external_mini_cluster.cc:1428] Setting key 51fd18b62f4b22fc0e16dbf6fb071051
I20260808 00:42:36.631580 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:42:36.637943 28048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.636342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45214 (local address 127.20.105.126:40435)
0808 00:42:36.636432 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.636434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.636699 (+ 265us) server_negotiation.cc:407] Connection header received
0808 00:42:36.636736 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.636737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.636744 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.636796 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.637498 (+ 702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.637500 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.637500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.637505 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.637524 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.637529 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.637589 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.637665 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.637681 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 00:42:36.640395 27983 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45214:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:36.642731 27983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:36.648108 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.645755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39833 (local address 127.0.0.1:60650)
0808 00:42:36.646091 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.646103 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:36.646164 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:36.646778 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:36.646785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:36.646795 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:36.647602 (+ 807us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:42:36.647602 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:36.647609 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:42:36.647619 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:36.647869 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:36.647877 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:36.647880 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:36.647976 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 00:42:36.647993 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":86,"threads_started":1}
I20260808 00:42:36.648130 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.646377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60650 (local address 127.20.105.66:39833)
0808 00:42:36.646578 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.646582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.646595 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:42:36.646628 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.646631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.646643 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.647091 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.647707 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.647716 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.647716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.647721 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.647770 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.647792 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.647864 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.647986 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.648003 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":126,"threads_started":1}
I20260808 00:42:36.648236 28646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.645719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:46147 (local address 127.0.0.1:38406)
0808 00:42:36.645898 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.645958 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:36.646055 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:36.646815 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:36.646816 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:36.646818 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:36.647506 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:42:36.647506 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:36.647530 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:42:36.647596 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:36.648107 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:36.648109 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:36.648109 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 00:42:36.648144 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 00:42:36.648159 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260808 00:42:36.648336 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.646079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33186 (local address 127.20.105.65:46057)
0808 00:42:36.646313 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.646320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.646335 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:42:36.646687 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.646695 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.646707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.646791 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.647884 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.647894 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.647894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.647900 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.647946 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.648009 (+ 63us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.648080 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.648206 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.648223 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":150,"threads_started":1}
I20260808 00:42:36.648542 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.645760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46057 (local address 127.0.0.1:33186)
0808 00:42:36.646258 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.646267 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:36.646566 (+ 299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:36.647240 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:36.647241 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:36.647243 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:36.647748 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:42:36.647748 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:36.647758 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:42:36.647773 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:36.648127 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:36.648128 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:36.648129 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:42:36.648459 (+ 330us) client_negotiation.cc:240] Negotiation successful
0808 00:42:36.648475 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"spinlock_wait_cycles":1280,"thread_start_us":83,"threads_started":1}
I20260808 00:42:36.648694 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.646090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38406 (local address 127.20.105.67:46147)
0808 00:42:36.646648 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.646653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.646663 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:42:36.646697 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.646699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.646708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.646797 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.647919 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.647923 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.647923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.647927 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.647958 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.647973 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.648543 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.648571 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.648591 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":122,"threads_started":1}
I20260808 00:42:36.649566 28542 tablet_service.cc:1467] Processing CreateTablet for tablet 546f8716961b4363a8d51057bb21b129 (DEFAULT_TABLE table=test-table [id=4118120fc0114ca6a3fe2a0564269688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:36.649555 28168 tablet_service.cc:1467] Processing CreateTablet for tablet 546f8716961b4363a8d51057bb21b129 (DEFAULT_TABLE table=test-table [id=4118120fc0114ca6a3fe2a0564269688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:36.649573 28352 tablet_service.cc:1467] Processing CreateTablet for tablet 546f8716961b4363a8d51057bb21b129 (DEFAULT_TABLE table=test-table [id=4118120fc0114ca6a3fe2a0564269688]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:36.650031 28542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546f8716961b4363a8d51057bb21b129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:36.650031 28168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546f8716961b4363a8d51057bb21b129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:36.650043 28352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 546f8716961b4363a8d51057bb21b129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:36.653267 28652 tablet_bootstrap.cc:492] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: Bootstrap starting.
I20260808 00:42:36.654065 28653 tablet_bootstrap.cc:492] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b: Bootstrap starting.
I20260808 00:42:36.654352 28652 tablet_bootstrap.cc:654] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:36.654772 28652 log.cc:824] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:36.654953 28653 tablet_bootstrap.cc:654] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:36.655295 28653 log.cc:824] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:36.655766 28654 tablet_bootstrap.cc:492] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: Bootstrap starting.
I20260808 00:42:36.655838 28652 tablet_bootstrap.cc:492] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: No bootstrap required, opened a new log
I20260808 00:42:36.655910 28652 ts_tablet_manager.cc:1397] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:42:36.656193 28653 tablet_bootstrap.cc:492] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b: No bootstrap required, opened a new log
I20260808 00:42:36.656282 28653 ts_tablet_manager.cc:1397] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:36.656570 28654 tablet_bootstrap.cc:654] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:36.657152 28654 log.cc:824] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:36.658048 28654 tablet_bootstrap.cc:492] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: No bootstrap required, opened a new log
I20260808 00:42:36.658130 28654 ts_tablet_manager.cc:1397] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:36.659031 28652 raft_consensus.cc:348] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.659248 28652 raft_consensus.cc:374] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:36.659147 28653 raft_consensus.cc:348] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.659296 28652 raft_consensus.cc:729] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37be15b2c1174b94aa3645054baf074a, State: Initialized, Role: FOLLOWER
I20260808 00:42:36.659312 28653 raft_consensus.cc:374] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:36.659343 28653 raft_consensus.cc:729] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f622aa2b443144758b58044f5a93e08b, State: Initialized, Role: FOLLOWER
I20260808 00:42:36.659480 28653 consensus_queue.cc:260] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.659488 28652 consensus_queue.cc:260] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.659812 28652 ts_tablet_manager.cc:1428] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:42:36.659816 28653 ts_tablet_manager.cc:1428] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:42:36.659852 28261 heartbeater.cc:503] Master 127.20.105.126:40435 was elected leader, sending a full tablet report...
I20260808 00:42:36.659996 28635 heartbeater.cc:503] Master 127.20.105.126:40435 was elected leader, sending a full tablet report...
I20260808 00:42:36.661664 28654 raft_consensus.cc:348] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.661834 28654 raft_consensus.cc:374] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:36.661864 28654 raft_consensus.cc:729] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd0598086e444203b0bf7562244e08ee, State: Initialized, Role: FOLLOWER
I20260808 00:42:36.661999 28654 consensus_queue.cc:260] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.662284 28654 ts_tablet_manager.cc:1428] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:42:36.662335 28444 heartbeater.cc:503] Master 127.20.105.126:40435 was elected leader, sending a full tablet report...
W20260808 00:42:36.726739 28445 tablet.cc:2369] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:36.743265 28661 raft_consensus.cc:482] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:36.743510 28661 raft_consensus.cc:504] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.744166 28661 leader_election.cc:288] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 37be15b2c1174b94aa3645054baf074a (127.20.105.67:46147), bd0598086e444203b0bf7562244e08ee (127.20.105.66:39833)
I20260808 00:42:36.745707 28263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.744377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39833 (local address 127.20.105.65:43747)
0808 00:42:36.744464 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.744478 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:36.744527 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:36.744661 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:36.744662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:36.744664 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:36.745266 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:42:36.745268 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:36.745279 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:42:36.745299 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:36.745523 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:36.745524 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:36.745525 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:42:36.745595 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:42:36.745612 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 00:42:36.745744 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.744473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:43747 (local address 127.20.105.66:39833)
0808 00:42:36.744539 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.744542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.744554 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:36.744592 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.744593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.744601 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.744657 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.745400 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.745402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.745402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.745408 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.745433 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.745439 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.745511 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.745599 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.745623 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:42:36.745823 28669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.744383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:46147 (local address 127.20.105.65:33985)
0808 00:42:36.744751 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.744762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:36.744858 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:36.745042 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:36.745044 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:36.745058 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:36.745433 (+ 375us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:42:36.745433 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:36.745442 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:42:36.745454 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:36.745663 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:36.745664 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:36.745664 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 00:42:36.745718 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:42:36.745732 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":133,"thread_start_us":174,"threads_started":1}
I20260808 00:42:36.745821 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.744511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:33985 (local address 127.20.105.67:46147)
0808 00:42:36.744577 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.744580 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.744849 (+ 269us) server_negotiation.cc:407] Connection header received
0808 00:42:36.744929 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.744931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.744938 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.745031 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.745551 (+ 520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.745553 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.745554 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.745560 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.745587 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.745593 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.745652 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.745717 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.745732 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:42:36.746191 28573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546f8716961b4363a8d51057bb21b129" candidate_uuid: "f622aa2b443144758b58044f5a93e08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37be15b2c1174b94aa3645054baf074a" is_pre_election: true
I20260808 00:42:36.746402 28573 raft_consensus.cc:2454] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f622aa2b443144758b58044f5a93e08b in term 0.
I20260808 00:42:36.746567 28377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546f8716961b4363a8d51057bb21b129" candidate_uuid: "f622aa2b443144758b58044f5a93e08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0598086e444203b0bf7562244e08ee" is_pre_election: true
I20260808 00:42:36.746673 28097 leader_election.cc:302] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37be15b2c1174b94aa3645054baf074a, f622aa2b443144758b58044f5a93e08b; no voters:
I20260808 00:42:36.746788 28377 raft_consensus.cc:2454] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f622aa2b443144758b58044f5a93e08b in term 0.
I20260808 00:42:36.746815 28661 raft_consensus.cc:2781] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:36.746887 28661 raft_consensus.cc:482] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:36.746927 28661 raft_consensus.cc:3037] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 0 FOLLOWER]: Advancing to term 1
W20260808 00:42:36.822016 28262 tablet.cc:2369] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:36.830730 28661 raft_consensus.cc:504] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.831112 28661 leader_election.cc:288] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [CANDIDATE]: Term 1 election: Requested vote from peers 37be15b2c1174b94aa3645054baf074a (127.20.105.67:46147), bd0598086e444203b0bf7562244e08ee (127.20.105.66:39833)
I20260808 00:42:36.831483 28573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546f8716961b4363a8d51057bb21b129" candidate_uuid: "f622aa2b443144758b58044f5a93e08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37be15b2c1174b94aa3645054baf074a"
I20260808 00:42:36.831493 28377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "546f8716961b4363a8d51057bb21b129" candidate_uuid: "f622aa2b443144758b58044f5a93e08b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd0598086e444203b0bf7562244e08ee"
I20260808 00:42:36.831625 28377 raft_consensus.cc:3037] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:36.831626 28573 raft_consensus.cc:3037] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:36.833021 28377 raft_consensus.cc:2454] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f622aa2b443144758b58044f5a93e08b in term 1.
I20260808 00:42:36.833021 28573 raft_consensus.cc:2454] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f622aa2b443144758b58044f5a93e08b in term 1.
I20260808 00:42:36.833326 28096 leader_election.cc:302] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd0598086e444203b0bf7562244e08ee, f622aa2b443144758b58044f5a93e08b; no voters:
I20260808 00:42:36.833491 28661 raft_consensus.cc:2781] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:36.833719 28661 raft_consensus.cc:686] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 1 LEADER]: Becoming Leader. State: Replica: f622aa2b443144758b58044f5a93e08b, State: Running, Role: LEADER
I20260808 00:42:36.833863 28661 consensus_queue.cc:237] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } }
I20260808 00:42:36.835283 27980 catalog_manager.cc:5495] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b reported cstate change: term changed from 0 to 1, leader changed from <none> to f622aa2b443144758b58044f5a93e08b (127.20.105.65). New cstate: current_term: 1 leader_uuid: "f622aa2b443144758b58044f5a93e08b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f622aa2b443144758b58044f5a93e08b" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 } health_report { overall_health: UNKNOWN } } }
W20260808 00:42:36.873214 28636 tablet.cc:2369] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:36.883033 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.881761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.20.105.65:46057)
0808 00:42:36.881842 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:36.881844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:36.881854 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:42:36.881932 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:36.881933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:36.881954 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:36.882009 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:36.882740 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:36.882741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:36.882742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:36.882746 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:36.882767 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:36.882771 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:36.882823 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:36.882885 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:42:36.882899 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:42:36.885610 28377 raft_consensus.cc:1261] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 1 FOLLOWER]: Refusing update from remote peer f622aa2b443144758b58044f5a93e08b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:36.885610 28573 raft_consensus.cc:1261] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 1 FOLLOWER]: Refusing update from remote peer f622aa2b443144758b58044f5a93e08b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:36.886080 28671 consensus_queue.cc:1035] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [LEADER]: Connected to new peer: Peer: permanent_uuid: "37be15b2c1174b94aa3645054baf074a" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 46147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:36.886258 28661 consensus_queue.cc:1035] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd0598086e444203b0bf7562244e08ee" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:36.889549 28685 mvcc.cc:205] Tried to move back new op lower bound from 7316069404199985152 to 7316069403992522752. Current Snapshot: MvccSnapshot[applied={T|T < 7316069404199985152}]
I20260808 00:42:36.889717 28683 mvcc.cc:205] Tried to move back new op lower bound from 7316069404199985152 to 7316069403992522752. Current Snapshot: MvccSnapshot[applied={T|T < 7316069404199985152}]
I20260808 00:42:36.890344 28681 mvcc.cc:205] Tried to move back new op lower bound from 7316069404199985152 to 7316069403992522752. Current Snapshot: MvccSnapshot[applied={T|T < 7316069404199985152 or (T in {7316069404199985152})}]
I20260808 00:42:36.892405 27980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:36.892525 27980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45214:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:36.893579 27980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e31bf3b5d3a4bc78fe83574b6d7e296: Sending DeleteTablet for 3 replicas of tablet 546f8716961b4363a8d51057bb21b129
I20260808 00:42:36.894017 28168 tablet_service.cc:1514] Processing DeleteTablet for tablet 546f8716961b4363a8d51057bb21b129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:36 UTC) from {username='slave'} at 127.0.0.1:33186
I20260808 00:42:36.894289 28701 tablet_replica.cc:330] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b: stopping tablet replica
I20260808 00:42:36.894277 28542 tablet_service.cc:1514] Processing DeleteTablet for tablet 546f8716961b4363a8d51057bb21b129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:36 UTC) from {username='slave'} at 127.0.0.1:38406
I20260808 00:42:36.894457 28701 raft_consensus.cc:2229] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:36.894513 28704 tablet_replica.cc:330] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: stopping tablet replica
I20260808 00:42:36.894645 28704 raft_consensus.cc:2229] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:36.894788 28704 raft_consensus.cc:2258] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:36.894783 28701 raft_consensus.cc:2258] T 546f8716961b4363a8d51057bb21b129 P f622aa2b443144758b58044f5a93e08b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:36.895097 28352 tablet_service.cc:1514] Processing DeleteTablet for tablet 546f8716961b4363a8d51057bb21b129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:36 UTC) from {username='slave'} at 127.0.0.1:60650
I20260808 00:42:36.895368 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 28077
I20260808 00:42:36.895411 28704 ts_tablet_manager.cc:1905] T 546f8716961b4363a8d51057bb21b129 P 37be15b2c1174b94aa3645054baf074a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:36.910243 28706 tablet_replica.cc:330] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: stopping tablet replica
I20260808 00:42:36.910526 28706 raft_consensus.cc:2229] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:36.910748 28706 raft_consensus.cc:2258] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:36.911679 28706 ts_tablet_manager.cc:1905] T 546f8716961b4363a8d51057bb21b129 P bd0598086e444203b0bf7562244e08ee: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 00:42:36.918562 27961 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:46057 (error 108)
I20260808 00:42:36.918846 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 28264
I20260808 00:42:36.922585 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.922358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46057 (local address 127.0.0.1:33212)
0808 00:42:36.922425 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.922511 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 00:42:36.924634 27961 catalog_manager.cc:4584] TS f622aa2b443144758b58044f5a93e08b (127.20.105.65:46057): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 546f8716961b4363a8d51057bb21b129: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46057: connect: Connection refused (error 111)
I20260808 00:42:36.937443 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 28447
I20260808 00:42:36.937611 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.937393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39833 (local address 127.0.0.1:60664)
0808 00:42:36.937486 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.937544 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.66:39833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 00:42:36.979102 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.978840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46057 (local address 127.0.0.1:33224)
0808 00:42:36.978972 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.979035 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 00:42:36.979779 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.979625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46057 (local address 127.0.0.1:33232)
0808 00:42:36.979710 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.979746 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 00:42:36.984818 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.984658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39833 (local address 127.0.0.1:60676)
0808 00:42:36.984725 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.984768 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.66:39833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 00:42:36.985281 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:36.985164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39833 (local address 127.0.0.1:60688)
0808 00:42:36.985218 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:36.985246 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.66:39833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 00:42:37.027814 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.027507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46057 (local address 127.0.0.1:33240)
0808 00:42:37.027657 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:37.027727 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260808 00:42:37.028374 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.028212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46057 (local address 127.0.0.1:33252)
0808 00:42:37.028280 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:37.028321 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:42:37.033392 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 27936
I20260808 00:42:37.033427 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.033255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:46147 (local address 127.0.0.1:38422)
0808 00:42:37.033352 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:37.033384 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.67:46147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
2026-08-08T00:42:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1177 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 00:42:37.067894 20901 mini_oidc.cc:152] Starting JWKS server
I20260808 00:42:37.069550 20901 webserver.cc:460] Webserver started at https://127.0.0.1:35381/ using document root <none> and password file <none>
I20260808 00:42:37.069700 20901 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:42:37.069926 20901 webserver.cc:460] Webserver started at http://127.0.0.1:39753/ using document root <none> and password file <none>
2026-08-08T00:42:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:37Z Disabled control of system clock
I20260808 00:42:37.083333 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:36311
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:46277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:36311
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35381/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/testchainca.cert with env {}
W20260808 00:42:37.189652 28717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:37.190056 28717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:37.190099 28717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:37.191574 28717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:37.191632 28717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:37.191644 28717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:37.191658 28717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:37.191670 28717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:42:37.191684 28717 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:42:37.191743 28717 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35381/jwks.json
I20260808 00:42:37.193712 28717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:36311
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35381/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:37.194070 28717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:37.194387 28717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:37.200577 28722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:37.200590 28727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:37.200579 28729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:37.200941 28717 server_base.cc:1034] running on GCE node
I20260808 00:42:37.201315 28717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:37.201756 28717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:37.202927 28717 hybrid_clock.cc:648] HybridClock initialized: now 1786149757202901 us; error 52 us; skew 500 ppm
I20260808 00:42:37.204766 28717 webserver.cc:460] Webserver started at http://127.20.105.126:34859/ using document root <none> and password file <none>
I20260808 00:42:37.205149 28717 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:37.205209 28717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:37.205374 28717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:37.206715 28717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "02c6f3acefe4d507a63c44f2406267c1"
server_key_iv: "ee23bdc35b456e458d16206d571e2c39"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.207185 28717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "02c6f3acefe4d507a63c44f2406267c1"
server_key_iv: "ee23bdc35b456e458d16206d571e2c39"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.209146 28717 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260808 00:42:37.210233 28737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:37.210464 28717 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:37.210608 28717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "02c6f3acefe4d507a63c44f2406267c1"
server_key_iv: "ee23bdc35b456e458d16206d571e2c39"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.210712 28717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:37.230764 28717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:37.231357 28717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:37.231583 28717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:37.237617 28717 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:36311
I20260808 00:42:37.237668 28808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:36311 every 8 connection(s)
I20260808 00:42:37.238217 28717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:37.239621 28809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:37.240091 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 28717
I20260808 00:42:37.240226 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:37.240433 20901 external_mini_cluster.cc:1428] Setting key 28ecd986c5ceff2d8c166ed86a484deb
I20260808 00:42:37.246187 28809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3: Bootstrap starting.
I20260808 00:42:37.247346 28809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:37.247826 28809 log.cc:824] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:37.249073 28809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3: No bootstrap required, opened a new log
I20260808 00:42:37.250656 28812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.241145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35644 (local address 127.20.105.126:36311)
0808 00:42:37.241446 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:37.241458 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:37.241498 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:42:37.242601 (+ 1103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:37.242632 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:37.242765 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:37.242994 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:37.243364 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.245176 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.246585 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.247448 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.247625 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.249533 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:37.249537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:37.249537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:37.249750 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:37.249795 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:37.249809 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:37.249895 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:37.250171 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 00:42:37.250290 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":116,"threads_started":1}
I20260808 00:42:37.251539 28809 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36311 } }
I20260808 00:42:37.251771 28809 raft_consensus.cc:374] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:37.251812 28809 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa8d6c7a2da04fa7a4ea4b09749ffad3, State: Initialized, Role: FOLLOWER
I20260808 00:42:37.251997 28809 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36311 } }
I20260808 00:42:37.252135 28809 raft_consensus.cc:388] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:37.252171 28809 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:37.252223 28809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:37.252671 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:42:37.341130 28809 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36311 } }
I20260808 00:42:37.341445 28809 leader_election.cc:302] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aa8d6c7a2da04fa7a4ea4b09749ffad3; no voters:
I20260808 00:42:37.341786 28809 leader_election.cc:288] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:37.341887 28814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:37.342096 28814 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [term 1 LEADER]: Becoming Leader. State: Replica: aa8d6c7a2da04fa7a4ea4b09749ffad3, State: Running, Role: LEADER
I20260808 00:42:37.342280 28809 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:37.342242 28814 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36311 } }
I20260808 00:42:37.342939 28817 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa8d6c7a2da04fa7a4ea4b09749ffad3. Latest consensus state: current_term: 1 leader_uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36311 } } }
I20260808 00:42:37.342957 28815 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aa8d6c7a2da04fa7a4ea4b09749ffad3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36311 } } }
I20260808 00:42:37.343065 28817 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:37.343091 28815 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:37.343313 28825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:37.344359 28825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:37.347687 28825 catalog_manager.cc:1347] Generated new cluster ID: 01b4c2c424a24a92bd609be3eccd1ad9
I20260808 00:42:37.347743 28825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:37.362251 28825 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:37.362923 28825 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:37.369333 28825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa8d6c7a2da04fa7a4ea4b09749ffad3: Generated new TSK 0
I20260808 00:42:37.369634 28825 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 00:42:37.399022 28858 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 00:42:37.399201 28858 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 00:42:37.399477 28812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.380756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.20.105.126:36311)
0808 00:42:37.380824 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:37.380827 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:37.380982 (+ 155us) server_negotiation.cc:407] Connection header received
0808 00:42:37.381141 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:37.381142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:37.381159 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:37.381254 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:42:37.381429 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.382319 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.383136 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.383462 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.383537 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.383785 (+ 248us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:42:37.399144 (+ 15359us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 00:42:37.399381 (+ 237us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35650: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 00:42:37.399551 28812 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35650: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 00:42:37.400242 28857 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:37.380656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:36311 (local address 127.0.0.1:35650)
0808 00:42:37.380885 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:37.380904 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:37.380996 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:37.381273 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:37.381274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:37.381276 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:42:37.381356 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:37.381359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.382406 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.382407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:37.383037 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:37.383039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.383555 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.383556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:37.383609 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.383635 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:42:37.399479 (+ 15844us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 00:42:37.399679 (+ 200us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.105.126:36311: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.105.126:36311: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35381/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 00:42:37.400874 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 28717
2026-08-08T00:42:37Z chronyd exiting
I20260808 00:42:37.456386 20901 test_util.cc:175] -----------------------------------------------
I20260808 00:42:37.456533 20901 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149729697709-20901-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (390 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 00:42:37.457639 20901 mini_oidc.cc:152] Starting JWKS server
I20260808 00:42:37.458470 20901 webserver.cc:460] Webserver started at https://127.0.0.1:44307/ using document root <none> and password file <none>
I20260808 00:42:37.458540 20901 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:42:37.458698 20901 webserver.cc:460] Webserver started at http://127.0.0.1:42351/ using document root <none> and password file <none>
2026-08-08T00:42:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:37Z Disabled control of system clock
I20260808 00:42:37.470638 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44457
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:39995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:44457
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44307/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/testchainca.cert with env {}
W20260808 00:42:37.569741 28870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:37.570037 28870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:37.570063 28870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:37.571379 28870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:37.571429 28870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:37.571442 28870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:37.571455 28870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:37.571468 28870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:42:37.571523 28870 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:42:37.571548 28870 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44307/jwks.json
I20260808 00:42:37.573277 28870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:39995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:44457
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44307/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:37.573612 28870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:37.573881 28870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:37.579754 28877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:37.579769 28876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:37.579804 28880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:37.580571 28870 server_base.cc:1034] running on GCE node
I20260808 00:42:37.580889 28870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:37.581244 28870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:37.582471 28870 hybrid_clock.cc:648] HybridClock initialized: now 1786149757582453 us; error 51 us; skew 500 ppm
I20260808 00:42:37.588551 28870 webserver.cc:460] Webserver started at http://127.20.105.126:43921/ using document root <none> and password file <none>
I20260808 00:42:37.588932 28870 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:37.589015 28870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:37.589212 28870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:37.590649 28870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "ee8fc5936fda4024a39c28e65bda15ec"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "ce2454ad90fdc00edb4c230e97d43d7b"
server_key_iv: "8f69716bc8472d51cc05e55208f56a23"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.591144 28870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "ee8fc5936fda4024a39c28e65bda15ec"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "ce2454ad90fdc00edb4c230e97d43d7b"
server_key_iv: "8f69716bc8472d51cc05e55208f56a23"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.592866 28870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260808 00:42:37.593878 28888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:37.594137 28870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:37.594264 28870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "ee8fc5936fda4024a39c28e65bda15ec"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "ce2454ad90fdc00edb4c230e97d43d7b"
server_key_iv: "8f69716bc8472d51cc05e55208f56a23"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.594352 28870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:37.604740 28870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:37.605194 28870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:37.605379 28870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:37.610941 28870 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:44457
I20260808 00:42:37.611016 28967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:44457 every 8 connection(s)
I20260808 00:42:37.611462 28870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:37.612473 28968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:37.616168 28968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec: Bootstrap starting.
I20260808 00:42:37.616921 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 28870
I20260808 00:42:37.617028 28968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:37.617069 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:37.617311 20901 external_mini_cluster.cc:1428] Setting key e40e7e87bad7ea24f1660924bdfe1751
I20260808 00:42:37.617461 28968 log.cc:824] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:37.618474 28968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec: No bootstrap required, opened a new log
I20260808 00:42:37.621021 28968 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8fc5936fda4024a39c28e65bda15ec" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44457 } }
I20260808 00:42:37.621232 28968 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:37.621287 28968 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8fc5936fda4024a39c28e65bda15ec, State: Initialized, Role: FOLLOWER
I20260808 00:42:37.621500 28968 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8fc5936fda4024a39c28e65bda15ec" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44457 } }
I20260808 00:42:37.621613 28968 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:37.621663 28968 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:37.621719 28968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:37.622465 28968 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8fc5936fda4024a39c28e65bda15ec" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44457 } }
I20260808 00:42:37.622608 28968 leader_election.cc:302] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee8fc5936fda4024a39c28e65bda15ec; no voters:
I20260808 00:42:37.622822 28968 leader_election.cc:288] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:37.622879 28978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:37.623153 28968 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:37.623327 28978 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [term 1 LEADER]: Becoming Leader. State: Replica: ee8fc5936fda4024a39c28e65bda15ec, State: Running, Role: LEADER
I20260808 00:42:37.623452 28978 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8fc5936fda4024a39c28e65bda15ec" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44457 } }
I20260808 00:42:37.623916 28980 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee8fc5936fda4024a39c28e65bda15ec. Latest consensus state: current_term: 1 leader_uuid: "ee8fc5936fda4024a39c28e65bda15ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8fc5936fda4024a39c28e65bda15ec" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44457 } } }
I20260808 00:42:37.624049 28980 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:37.624253 28979 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee8fc5936fda4024a39c28e65bda15ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee8fc5936fda4024a39c28e65bda15ec" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44457 } } }
I20260808 00:42:37.624321 28979 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:37.624361 28976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.618066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.20.105.126:44457)
0808 00:42:37.618330 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:37.618345 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:37.618376 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:42:37.619003 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:37.619013 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:37.619093 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:37.619265 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:37.619541 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.620628 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.621689 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.622204 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.622366 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.623256 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:37.623261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:37.623261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:37.623535 (+ 274us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:37.623587 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:37.623604 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:37.623676 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:37.623981 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 00:42:37.624123 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":95,"threads_started":1}
I20260808 00:42:37.624490 28987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:37.625116 28987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:37.625630 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:42:37.627061 28987 catalog_manager.cc:1347] Generated new cluster ID: 600206c1f7694ad08b165dc5d315e60f
I20260808 00:42:37.627116 28987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:37.633222 28987 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:37.634006 28987 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:37.646948 28987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee8fc5936fda4024a39c28e65bda15ec: Generated new TSK 0
I20260808 00:42:37.647321 28987 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 00:42:37.670991 29016 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 00:42:37.671166 29016 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 00:42:37.671597 28976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.654981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.20.105.126:44457)
0808 00:42:37.655047 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:37.655050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:37.655117 (+ 67us) server_negotiation.cc:407] Connection header received
0808 00:42:37.655203 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:37.655204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:37.655218 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:37.655322 (+ 104us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:42:37.655597 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.656497 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.657710 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.658068 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.658149 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.658389 (+ 240us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:42:37.671264 (+ 12875us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 00:42:37.671488 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 00:42:37.671679 28976 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38054: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 00:42:37.671694 29015 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:37.654855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44457 (local address 127.0.0.1:38054)
0808 00:42:37.655043 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:37.655061 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:37.655130 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:37.655365 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:37.655366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:37.655368 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:42:37.655475 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:37.655478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.656617 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.656619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:37.657585 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:37.657589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.658178 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.658180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:37.658245 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.658253 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:42:37.671429 (+ 13176us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 00:42:37.671570 (+ 141us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.105.126:44457: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44307/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":124,"threads_started":1}
I20260808 00:42:37.672183 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 28870
2026-08-08T00:42:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (265 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 00:42:37.723511 20901 mini_oidc.cc:152] Starting JWKS server
I20260808 00:42:37.724566 20901 webserver.cc:460] Webserver started at https://127.0.0.1:44397/ using document root <none> and password file <none>
I20260808 00:42:37.724654 20901 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:42:37.724859 20901 webserver.cc:460] Webserver started at http://127.0.0.1:46519/ using document root <none> and password file <none>
2026-08-08T00:42:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:37Z Disabled control of system clock
I20260808 00:42:37.738121 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:42573
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:34705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:42573
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44397/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 00:42:37.836767 29026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:37.837080 29026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:37.837110 29026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:37.838649 29026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:37.838711 29026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:37.838725 29026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:37.838739 29026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:37.838752 29026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:42:37.838765 29026 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:42:37.838830 29026 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44397/jwks.json
I20260808 00:42:37.840538 29026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:42573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:42573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44397/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:37.840917 29026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:37.841331 29026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:37.847365 29035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:37.847366 29038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:37.847486 29036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:37.847849 29026 server_base.cc:1034] running on GCE node
I20260808 00:42:37.848264 29026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:37.848644 29026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:37.849804 29026 hybrid_clock.cc:648] HybridClock initialized: now 1786149757849779 us; error 41 us; skew 500 ppm
I20260808 00:42:37.851452 29026 webserver.cc:460] Webserver started at http://127.20.105.126:45617/ using document root <none> and password file <none>
I20260808 00:42:37.851785 29026 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:37.851852 29026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:37.852077 29026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:37.853566 29026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "982c9c2f06564512a7d8ede058fe98b2"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "f450e4117aaf2a8c9b3b1ccdcb30e75e"
server_key_iv: "78eea7e7920760701434c29bb9614086"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.854046 29026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "982c9c2f06564512a7d8ede058fe98b2"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "f450e4117aaf2a8c9b3b1ccdcb30e75e"
server_key_iv: "78eea7e7920760701434c29bb9614086"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.855899 29026 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 00:42:37.857024 29046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:37.857267 29026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:37.857390 29026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "982c9c2f06564512a7d8ede058fe98b2"
format_stamp: "Formatted at 2026-08-08 00:42:37 on dist-test-slave-kvfs"
server_key: "f450e4117aaf2a8c9b3b1ccdcb30e75e"
server_key_iv: "78eea7e7920760701434c29bb9614086"
server_key_version: "encryptionkey@0"
I20260808 00:42:37.857483 29026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:37.866454 29026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:37.866849 29026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:37.867022 29026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:37.872340 29026 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:42573
I20260808 00:42:37.872418 29123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:42573 every 8 connection(s)
I20260808 00:42:37.873032 29026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:37.874100 29124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:37.875131 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 29026
I20260808 00:42:37.875275 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:37.875491 20901 external_mini_cluster.cc:1428] Setting key de7ace3b508500a6b11136e7e11acd74
I20260808 00:42:37.878314 29124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2: Bootstrap starting.
I20260808 00:42:37.879448 29124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:37.879844 29124 log.cc:824] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:37.880968 29124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2: No bootstrap required, opened a new log
I20260808 00:42:37.882397 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.876211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46184 (local address 127.20.105.126:42573)
0808 00:42:37.876443 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:37.876460 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:37.876493 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:42:37.876999 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:37.877006 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:37.877092 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:37.877236 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:37.877537 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.878638 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.880142 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.880627 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.880750 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.881579 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:37.881584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:37.881584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:37.881794 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:37.881836 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:37.881850 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:37.881905 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:37.882018 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:42:37.882126 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260808 00:42:37.883718 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:42:37.883771 29124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c9c2f06564512a7d8ede058fe98b2" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42573 } }
I20260808 00:42:37.883971 29124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:37.884003 29124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982c9c2f06564512a7d8ede058fe98b2, State: Initialized, Role: FOLLOWER
I20260808 00:42:37.884172 29124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c9c2f06564512a7d8ede058fe98b2" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42573 } }
I20260808 00:42:37.884256 29124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:37.884287 29124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:37.884337 29124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:37.885344 29124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c9c2f06564512a7d8ede058fe98b2" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42573 } }
I20260808 00:42:37.885455 29124 leader_election.cc:302] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 982c9c2f06564512a7d8ede058fe98b2; no voters:
I20260808 00:42:37.885633 29124 leader_election.cc:288] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:37.885710 29130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:37.885880 29130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [term 1 LEADER]: Becoming Leader. State: Replica: 982c9c2f06564512a7d8ede058fe98b2, State: Running, Role: LEADER
I20260808 00:42:37.885944 29124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:37.885985 29130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c9c2f06564512a7d8ede058fe98b2" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42573 } }
I20260808 00:42:37.886443 29132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "982c9c2f06564512a7d8ede058fe98b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c9c2f06564512a7d8ede058fe98b2" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42573 } } }
I20260808 00:42:37.886457 29134 sys_catalog.cc:455] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 982c9c2f06564512a7d8ede058fe98b2. Latest consensus state: current_term: 1 leader_uuid: "982c9c2f06564512a7d8ede058fe98b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "982c9c2f06564512a7d8ede058fe98b2" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42573 } } }
I20260808 00:42:37.886595 29134 sys_catalog.cc:458] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:37.886832 29132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:37.886936 29140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:37.887614 29140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:37.889489 29140 catalog_manager.cc:1347] Generated new cluster ID: 88951a9f9f5e41c38a24b8002f2e6c1f
I20260808 00:42:37.889575 29140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:37.896055 29140 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:37.896683 29140 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:37.898609 29140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 982c9c2f06564512a7d8ede058fe98b2: Generated new TSK 0
I20260808 00:42:37.898818 29140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:42:37.909145 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:37.904278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46196 (local address 127.20.105.126:42573)
0808 00:42:37.904358 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:37.904361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:37.904550 (+ 189us) server_negotiation.cc:407] Connection header received
0808 00:42:37.904611 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:37.904612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:37.904625 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:37.904699 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:42:37.904868 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.905401 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.906662 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.906906 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.906974 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.907212 (+ 238us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:42:37.908900 (+ 1688us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44397/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 00:42:37.909052 (+ 152us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46196: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44397/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 00:42:37.909222 29128 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46196: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44397/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 00:42:37.909375 29175 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:37.904220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:42573 (local address 127.0.0.1:46196)
0808 00:42:37.904441 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:37.904455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:37.904541 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:37.904716 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:37.904717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:37.904720 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:42:37.904802 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:37.904806 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.905501 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.905503 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:37.906554 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:37.906559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:37.906999 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:37.907001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:37.907081 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:37.907090 (+ 9us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:42:37.909020 (+ 1930us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44397/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 00:42:37.909155 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.105.126:42573: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44397/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":140,"threads_started":1}
I20260808 00:42:37.909677 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 29026
E20260808 00:42:37.935106 29177 webserver.cc:533] Webserver: sslize failed:
E20260808 00:42:37.935245 29177 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T00:42:37Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (243 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T00:42:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:37Z Disabled control of system clock
I20260808 00:42:37.978120 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44097
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:35671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:44097
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/testchainca.cert with env {}
W20260808 00:42:38.076563 29181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:38.076903 29181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:38.076948 29181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:38.078271 29181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:38.078344 29181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:38.078367 29181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:38.078390 29181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:38.078418 29181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:42:38.078450 29181 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:42:38.078474 29181 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 00:42:38.080235 29181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:44097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:38.080590 29181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:38.080860 29181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:38.086855 29189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:38.086875 29191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:38.086844 29195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:38.087321 29181 server_base.cc:1034] running on GCE node
I20260808 00:42:38.087611 29181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:38.087917 29181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:38.089071 29181 hybrid_clock.cc:648] HybridClock initialized: now 1786149758089059 us; error 78 us; skew 500 ppm
I20260808 00:42:38.090685 29181 webserver.cc:460] Webserver started at http://127.20.105.126:39949/ using document root <none> and password file <none>
I20260808 00:42:38.090969 29181 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:38.091024 29181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:38.091178 29181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:38.092612 29181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "1e10617edfe3400e952f392bfcfa3e9e"
format_stamp: "Formatted at 2026-08-08 00:42:38 on dist-test-slave-kvfs"
server_key: "0f57be9a0cd527c7b30cd3196c1b9cac"
server_key_iv: "e5953e5c372926ce6588cb62763099a9"
server_key_version: "encryptionkey@0"
I20260808 00:42:38.093075 29181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "1e10617edfe3400e952f392bfcfa3e9e"
format_stamp: "Formatted at 2026-08-08 00:42:38 on dist-test-slave-kvfs"
server_key: "0f57be9a0cd527c7b30cd3196c1b9cac"
server_key_iv: "e5953e5c372926ce6588cb62763099a9"
server_key_version: "encryptionkey@0"
I20260808 00:42:38.094720 29181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:38.095639 29203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:38.095878 29181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:38.095999 29181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "1e10617edfe3400e952f392bfcfa3e9e"
format_stamp: "Formatted at 2026-08-08 00:42:38 on dist-test-slave-kvfs"
server_key: "0f57be9a0cd527c7b30cd3196c1b9cac"
server_key_iv: "e5953e5c372926ce6588cb62763099a9"
server_key_version: "encryptionkey@0"
I20260808 00:42:38.096148 29181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:38.109412 29181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:38.109836 29181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:38.110018 29181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:38.115235 29181 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:44097
I20260808 00:42:38.115300 29279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:44097 every 8 connection(s)
I20260808 00:42:38.115741 29181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:38.117101 29282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:38.120121 29282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e: Bootstrap starting.
I20260808 00:42:38.120946 29282 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:38.121330 29282 log.cc:824] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:38.122290 29282 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e: No bootstrap required, opened a new log
I20260808 00:42:38.124197 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 29181
I20260808 00:42:38.124344 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:38.124346 29282 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e10617edfe3400e952f392bfcfa3e9e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44097 } }
I20260808 00:42:38.124568 29282 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:38.124575 20901 external_mini_cluster.cc:1428] Setting key 257d94b026ff0ded9926f9334631b686
I20260808 00:42:38.124617 29282 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e10617edfe3400e952f392bfcfa3e9e, State: Initialized, Role: FOLLOWER
I20260808 00:42:38.124821 29282 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e10617edfe3400e952f392bfcfa3e9e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44097 } }
I20260808 00:42:38.124938 29282 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:38.124984 29282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:38.125036 29282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:38.125773 29282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e10617edfe3400e952f392bfcfa3e9e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44097 } }
I20260808 00:42:38.125921 29282 leader_election.cc:302] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e10617edfe3400e952f392bfcfa3e9e; no voters:
I20260808 00:42:38.126137 29282 leader_election.cc:288] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:38.126171 29287 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:38.126312 29287 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [term 1 LEADER]: Becoming Leader. State: Replica: 1e10617edfe3400e952f392bfcfa3e9e, State: Running, Role: LEADER
I20260808 00:42:38.126415 29287 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e10617edfe3400e952f392bfcfa3e9e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44097 } }
I20260808 00:42:38.126493 29282 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:38.126856 29288 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e10617edfe3400e952f392bfcfa3e9e. Latest consensus state: current_term: 1 leader_uuid: "1e10617edfe3400e952f392bfcfa3e9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e10617edfe3400e952f392bfcfa3e9e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44097 } } }
I20260808 00:42:38.126956 29288 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:38.127132 29289 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e10617edfe3400e952f392bfcfa3e9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e10617edfe3400e952f392bfcfa3e9e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44097 } } }
I20260808 00:42:38.127223 29289 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:38.127709 29295 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:38.128520 29295 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:38.130946 29295 catalog_manager.cc:1347] Generated new cluster ID: 746ae16c74264caf962535d8bfa138f3
I20260808 00:42:38.131014 29295 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:38.132824 29286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:38.125150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57624 (local address 127.20.105.126:44097)
0808 00:42:38.125417 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:38.125427 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:38.125471 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:42:38.125617 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:38.125623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:38.125698 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:38.125861 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:38.126114 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:38.128324 (+ 2210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:38.129554 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:38.130624 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:38.130840 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:38.131711 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:38.131716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:38.131716 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:42:38.132007 (+ 291us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:38.132134 (+ 127us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:42:38.132153 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:38.132227 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:38.132343 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 00:42:38.132555 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":99,"threads_started":1}
I20260808 00:42:38.134316 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:42:38.138963 29295 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:38.139753 29295 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:38.148010 29295 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e10617edfe3400e952f392bfcfa3e9e: Generated new TSK 0
I20260808 00:42:38.148365 29295 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:42:38.608206 29286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:38.155775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.20.105.126:44097)
0808 00:42:38.155851 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:38.155854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:38.155991 (+ 137us) server_negotiation.cc:407] Connection header received
0808 00:42:38.156128 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:38.156129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:38.156141 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:38.156215 (+ 74us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:42:38.156395 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:38.156925 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:38.158167 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:38.158363 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:38.158422 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:38.158639 (+ 217us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:42:38.607737 (+449098us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 00:42:38.608011 (+ 274us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57634: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 00:42:38.608325 29286 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57634: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 00:42:38.608559 29321 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:38.155697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44097 (local address 127.0.0.1:57634)
0808 00:42:38.155900 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:38.155913 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:38.156010 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:38.156235 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:38.156236 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:38.156239 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:42:38.156322 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:38.156325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:38.157039 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:38.157040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:38.158071 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:38.158075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:38.158447 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:38.158448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:38.158513 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:38.158521 (+ 8us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:42:38.607972 (+449451us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 00:42:38.608265 (+ 293us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.105.126:44097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":121,"threads_started":1}
I20260808 00:42:38.609017 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 29181
2026-08-08T00:42:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (674 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (117 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (119 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:38 dist-test-slave-kvfs krb5kdc[29345](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:38 dist-test-slave-kvfs krb5kdc[29345](info): set up 2 sockets
Aug 08 00:42:38 dist-test-slave-kvfs krb5kdc[29345](info): commencing operation
krb5kdc: starting...
W20260808 00:42:40.172436 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.294s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:40 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:40.324142 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:38009
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:38009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:40.420682 29364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:40.421020 29364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:40.421066 29364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:40.422394 29364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:40.422499 29364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:40.422526 29364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:40.422549 29364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:40.422580 29364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:40.424381 29364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:38009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:38009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:40.424763 29364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:40.425081 29364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:40.431339 29371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:40.431383 29364 server_base.cc:1034] running on GCE node
W20260808 00:42:40.431345 29372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:40.431361 29375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:40.432134 29364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:40.432523 29364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:40.433712 29364 hybrid_clock.cc:648] HybridClock initialized: now 1786149760433689 us; error 43 us; skew 500 ppm
Aug 08 00:42:40 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:40.437081 29364 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:40.437723 29364 webserver.cc:460] Webserver started at http://127.20.105.126:45755/ using document root <none> and password file <none>
I20260808 00:42:40.438102 29364 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:40.438194 29364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:40.438397 29364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:40.439988 29364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "76c6d1c0dea6439b990c0954b9cb541b"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "882d5c440b4b934951be302827d0b654"
server_key_iv: "da1f7016449eaf992bb70d6caf64f08f"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.440562 29364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "76c6d1c0dea6439b990c0954b9cb541b"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "882d5c440b4b934951be302827d0b654"
server_key_iv: "da1f7016449eaf992bb70d6caf64f08f"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.442538 29364 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:40.443636 29384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:40.443912 29364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:40.444072 29364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "76c6d1c0dea6439b990c0954b9cb541b"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "882d5c440b4b934951be302827d0b654"
server_key_iv: "da1f7016449eaf992bb70d6caf64f08f"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.444190 29364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:40.462973 29364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:40.466229 29364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:40.466432 29364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:40.471704 29364 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:38009
I20260808 00:42:40.471745 29461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:38009 every 8 connection(s)
I20260808 00:42:40.472278 29364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:40.473433 29463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:40.476456 29463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b: Bootstrap starting.
I20260808 00:42:40.477257 29463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:40.477632 29463 log.cc:824] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:40.478523 29463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b: No bootstrap required, opened a new log
I20260808 00:42:40.480533 29463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c6d1c0dea6439b990c0954b9cb541b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38009 } }
I20260808 00:42:40.480718 29463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:40.480744 29463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c6d1c0dea6439b990c0954b9cb541b, State: Initialized, Role: FOLLOWER
I20260808 00:42:40.480942 29463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c6d1c0dea6439b990c0954b9cb541b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38009 } }
I20260808 00:42:40.481014 29463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:40.481066 29463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:40.481125 29463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:40.481796 29463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c6d1c0dea6439b990c0954b9cb541b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38009 } }
I20260808 00:42:40.481927 29463 leader_election.cc:302] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c6d1c0dea6439b990c0954b9cb541b; no voters:
I20260808 00:42:40.482159 29463 leader_election.cc:288] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:40.482491 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 29364
I20260808 00:42:40.482613 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:40.482833 20901 external_mini_cluster.cc:1428] Setting key a207766e2161b9637b941a020dfa9c7e
I20260808 00:42:40.483035 29467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:40.483345 29463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:40.483510 29467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [term 1 LEADER]: Becoming Leader. State: Replica: 76c6d1c0dea6439b990c0954b9cb541b, State: Running, Role: LEADER
I20260808 00:42:40.483867 29467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c6d1c0dea6439b990c0954b9cb541b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38009 } }
I20260808 00:42:40.485582 29469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c6d1c0dea6439b990c0954b9cb541b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c6d1c0dea6439b990c0954b9cb541b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38009 } } }
I20260808 00:42:40.485605 29471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c6d1c0dea6439b990c0954b9cb541b. Latest consensus state: current_term: 1 leader_uuid: "76c6d1c0dea6439b990c0954b9cb541b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c6d1c0dea6439b990c0954b9cb541b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38009 } } }
I20260808 00:42:40.485695 29469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:40.485711 29471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b [sys.catalog]: This master's current role is: LEADER
W20260808 00:42:40.486266 29489 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:42:40.486325 29489 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:42:40.486399 29491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:40.486718 29491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:40.488507 29491 catalog_manager.cc:1347] Generated new cluster ID: 94ede953a7b743cd863872a4f4b2db4a
I20260808 00:42:40.488569 29491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:42:40 dist-test-slave-kvfs krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:40.495915 29491 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:40.496660 29491 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:40.497787 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:40.483686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56350 (local address 127.20.105.126:38009)
0808 00:42:40.483850 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:40.483861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:40.483893 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:42:40.484078 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:40.484087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:40.484178 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:40.485430 (+ 1252us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:40.485578 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.487151 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.488330 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.488969 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.489090 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:40.492553 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:40.492559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:40.492560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:40.492575 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:40.495704 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:40.496260 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:40.496264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:40.496265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:40.496393 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:40.496656 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:40.496657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:40.496657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:40.497053 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:40.497247 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:40.497337 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 00:42:40.497522 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260808 00:42:40.501662 29491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c6d1c0dea6439b990c0954b9cb541b: Generated new TSK 0
I20260808 00:42:40.501889 29491 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:40.539963 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:38009
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:40.638427 29498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:40.638693 29498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:40.638720 29498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:40.640009 29498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:40.640113 29498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:40.640158 29498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:40.641834 29498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:38009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:40.642181 29498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:40.642482 29498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:40.648828 29506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:40.648818 29509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:40.648819 29504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:40.649127 29498 server_base.cc:1034] running on GCE node
I20260808 00:42:40.649500 29498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:40.649812 29498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:40.651026 29498 hybrid_clock.cc:648] HybridClock initialized: now 1786149760651007 us; error 49 us; skew 500 ppm
Aug 08 00:42:40 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:40.654358 29498 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:40.654948 29498 webserver.cc:460] Webserver started at http://127.20.105.65:43499/ using document root <none> and password file <none>
I20260808 00:42:40.655273 29498 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:40.655356 29498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:40.655539 29498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:40.657007 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "fac9007130754f24805046d0f3c27984"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "248fd7a6929c9fe7873bc54d65fe38d7"
server_key_iv: "9f46f58afeff5a1c03637cd7de3ad9d2"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.657450 29498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "fac9007130754f24805046d0f3c27984"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "248fd7a6929c9fe7873bc54d65fe38d7"
server_key_iv: "9f46f58afeff5a1c03637cd7de3ad9d2"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.659160 29498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:42:40.660149 29518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:40.660346 29498 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:40.660472 29498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "fac9007130754f24805046d0f3c27984"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "248fd7a6929c9fe7873bc54d65fe38d7"
server_key_iv: "9f46f58afeff5a1c03637cd7de3ad9d2"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.660560 29498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:40.679076 29498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:40.681665 29498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:40.681897 29498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:40.682226 29498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:40.682647 29498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:40.682708 29498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:40.682755 29498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:40.682826 29498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:40.690687 29498 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:44873
I20260808 00:42:40.690735 29700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:44873 every 8 connection(s)
I20260808 00:42:40.691181 29498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:40.698149 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 29498
I20260808 00:42:40.698271 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:40.698503 20901 external_mini_cluster.cc:1428] Setting key 0ea5fd8cb8b6b5cdad11ef674fd412fd
Aug 08 00:42:40 dist-test-slave-kvfs krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:40.703282 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:40.692190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:60423 (local address 127.20.105.126:38009)
0808 00:42:40.692287 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:40.692291 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:40.692517 (+ 226us) server_negotiation.cc:407] Connection header received
0808 00:42:40.693681 (+ 1164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:40.693683 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:40.693707 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:40.693775 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:40.694633 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.695235 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.696370 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.696663 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.696744 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:40.699821 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:40.699823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:40.699823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:40.699833 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:40.701613 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:40.702123 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:40.702124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:40.702125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:40.702198 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:40.702492 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:40.702493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:40.702493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:40.702640 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:40.702718 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:40.703100 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 00:42:40.703197 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:42:40.703621 29704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:40.692128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:38009 (local address 127.20.105.65:60423)
0808 00:42:40.692352 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:40.692407 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:40.693525 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:40.693874 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:40.693878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:40.694245 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:40.694500 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:40.694504 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.695331 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.695332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:40.696288 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:40.696290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.696769 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.696771 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:40.696967 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:40.697784 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:40.697800 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:40.699711 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:40.701731 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:40.701741 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:40.701744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:40.702017 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:40.702277 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:40.702278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:40.702279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:40.702414 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:40.702723 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:40.702732 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:40.702906 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 00:42:40.703041 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 00:42:40.703285 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260808 00:42:40.704341 29701 heartbeater.cc:347] Connected to a master server at 127.20.105.126:38009
I20260808 00:42:40.704478 29701 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:40.704738 29701 heartbeater.cc:511] Master 127.20.105.126:38009 requested a full tablet report, sending...
I20260808 00:42:40.705471 29409 ts_manager.cc:194] Registered new tserver with Master: fac9007130754f24805046d0f3c27984 (127.20.105.65:44873)
I20260808 00:42:40.706188 29409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:60423
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:40.731964 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:38009
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:40.831025 29707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:40.831287 29707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:40.831315 29707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:40.832643 29707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:40.832691 29707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:40.832719 29707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:40.834323 29707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:38009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:40.834581 29707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:40.834828 29707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:40.841439 29716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:40.841439 29713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:40.841459 29714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:40.841821 29707 server_base.cc:1034] running on GCE node
I20260808 00:42:40.842099 29707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:40.842367 29707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:40.843531 29707 hybrid_clock.cc:648] HybridClock initialized: now 1786149760843518 us; error 37 us; skew 500 ppm
Aug 08 00:42:40 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:40.847157 29707 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:40.847728 29707 webserver.cc:460] Webserver started at http://127.20.105.66:34065/ using document root <none> and password file <none>
I20260808 00:42:40.848172 29707 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:40.848238 29707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:40.848428 29707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:40.849946 29707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "bddad844bad84ae7978870ec47600e68"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "a83a7f29f4f6e833f8b3135df3d46ace"
server_key_iv: "6af68066c9545e3fad5c2222cf64f79e"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.850472 29707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "bddad844bad84ae7978870ec47600e68"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "a83a7f29f4f6e833f8b3135df3d46ace"
server_key_iv: "6af68066c9545e3fad5c2222cf64f79e"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.852522 29707 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 00:42:40.853654 29726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:40.854007 29707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:40.854147 29707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "bddad844bad84ae7978870ec47600e68"
format_stamp: "Formatted at 2026-08-08 00:42:40 on dist-test-slave-kvfs"
server_key: "a83a7f29f4f6e833f8b3135df3d46ace"
server_key_iv: "6af68066c9545e3fad5c2222cf64f79e"
server_key_version: "encryptionkey@0"
I20260808 00:42:40.854252 29707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:40.882699 29707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:40.885942 29707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:40.886127 29707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:40.886469 29707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:40.886869 29707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:40.886929 29707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:40.886969 29707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:40.887017 29707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:40.894166 29707 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:32851
I20260808 00:42:40.894224 29886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:32851 every 8 connection(s)
I20260808 00:42:40.894654 29707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:40.900712 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 29707
I20260808 00:42:40.900849 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:40.901125 20901 external_mini_cluster.cc:1428] Setting key 82105503dedcc219d2993977d9fe40e4
Aug 08 00:42:40 dist-test-slave-kvfs krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:40.907106 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:40.895688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:44667 (local address 127.20.105.126:38009)
0808 00:42:40.895793 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:40.895796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:40.896004 (+ 208us) server_negotiation.cc:407] Connection header received
0808 00:42:40.897193 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:40.897195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:40.897210 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:40.897284 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:40.898178 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.898790 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.899977 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.900230 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.900302 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:40.903411 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:40.903413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:40.903414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:40.903424 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:40.905389 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:40.905867 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:40.905868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:40.905869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:40.905982 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:40.906246 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:40.906247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:40.906247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:40.906447 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:40.906536 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:40.906883 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 00:42:40.906994 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 00:42:40.907348 29893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:40.895604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:38009 (local address 127.20.105.66:44667)
0808 00:42:40.895802 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:40.895855 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:40.897031 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:40.897410 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:40.897415 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:40.897757 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:40.898021 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:40.898027 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.898889 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.898889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:40.899832 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:40.899835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:40.900330 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:40.900331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:40.900530 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:40.901281 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:40.901294 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:40.903280 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:40.905489 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:40.905497 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:40.905500 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:40.905786 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:40.906060 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:40.906061 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:40.906061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:40.906179 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:40.906526 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:40.906535 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:40.906709 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 00:42:40.906839 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 00:42:40.907051 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":101,"threads_started":1}
I20260808 00:42:40.908077 29891 heartbeater.cc:347] Connected to a master server at 127.20.105.126:38009
I20260808 00:42:40.908226 29891 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:40.908529 29891 heartbeater.cc:511] Master 127.20.105.126:38009 requested a full tablet report, sending...
I20260808 00:42:40.909122 29409 ts_manager.cc:194] Registered new tserver with Master: bddad844bad84ae7978870ec47600e68 (127.20.105.66:32851)
I20260808 00:42:40.909729 29409 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:44667
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:40.936276 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:38009
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:41.037734 29897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:41.038033 29897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:41.038065 29897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:41.039497 29897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:41.039559 29897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:41.039592 29897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:41.041648 29897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:38009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:41.041929 29897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:41.042187 29897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:41.048467 29904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:41.048434 29907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:41.048457 29905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:41.049077 29897 server_base.cc:1034] running on GCE node
I20260808 00:42:41.049345 29897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:41.049680 29897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:41.050864 29897 hybrid_clock.cc:648] HybridClock initialized: now 1786149761050838 us; error 62 us; skew 500 ppm
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:41.054291 29897 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:41.054946 29897 webserver.cc:460] Webserver started at http://127.20.105.67:35173/ using document root <none> and password file <none>
I20260808 00:42:41.055260 29897 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:41.055342 29897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:41.055548 29897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:41.057088 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "9c03b436f46041eaab3249f0ea8248cf"
format_stamp: "Formatted at 2026-08-08 00:42:41 on dist-test-slave-kvfs"
server_key: "ce6b91a6f50fc44a72c50058dbe0849f"
server_key_iv: "0aa112578c3fa493ac5c6e6bab1814f2"
server_key_version: "encryptionkey@0"
I20260808 00:42:41.057627 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "9c03b436f46041eaab3249f0ea8248cf"
format_stamp: "Formatted at 2026-08-08 00:42:41 on dist-test-slave-kvfs"
server_key: "ce6b91a6f50fc44a72c50058dbe0849f"
server_key_iv: "0aa112578c3fa493ac5c6e6bab1814f2"
server_key_version: "encryptionkey@0"
I20260808 00:42:41.059585 29897 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 00:42:41.060691 29916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:41.060916 29897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:41.061074 29897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "9c03b436f46041eaab3249f0ea8248cf"
format_stamp: "Formatted at 2026-08-08 00:42:41 on dist-test-slave-kvfs"
server_key: "ce6b91a6f50fc44a72c50058dbe0849f"
server_key_iv: "0aa112578c3fa493ac5c6e6bab1814f2"
server_key_version: "encryptionkey@0"
I20260808 00:42:41.061175 29897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:41.070381 29897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:41.072858 29897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:41.073074 29897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:41.073374 29897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:41.073774 29897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:41.073846 29897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:41.073897 29897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:41.073927 29897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:41.081210 29897 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:42023
I20260808 00:42:41.081276 30071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:42023 every 8 connection(s)
I20260808 00:42:41.081681 29897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:41.084321 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 29897
I20260808 00:42:41.084460 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:41.084745 20901 external_mini_cluster.cc:1428] Setting key e441bb8cdf25ee6058ef2a72f1caaeb5
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:41.096063 29472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:41.082683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:40603 (local address 127.20.105.126:38009)
0808 00:42:41.082789 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:41.082792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:41.082990 (+ 198us) server_negotiation.cc:407] Connection header received
0808 00:42:41.084326 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:41.084328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:41.084344 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:41.084400 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:41.085449 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.086024 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.088238 (+ 2214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.088541 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.088634 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:41.091930 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:41.091932 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:41.091933 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:41.091960 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:41.094063 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:41.094644 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:41.094645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:41.094646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:41.094743 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:41.095121 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:41.095122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:41.095122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:41.095296 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:41.095367 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:41.095788 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 00:42:41.095898 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 00:42:41.096369 30074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:41.082541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:38009 (local address 127.20.105.67:40603)
0808 00:42:41.082810 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:41.082864 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:41.084184 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:41.084497 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:41.084502 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:41.084975 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:41.085337 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:41.085344 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.086130 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.086132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:41.087451 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:41.087453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.088636 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.088637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:41.088831 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:41.089601 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:41.089614 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:41.091806 (+ 2192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:41.094172 (+ 2366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:41.094181 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:41.094184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:41.094545 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:41.094828 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:41.094829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:41.094830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:41.095023 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:41.095379 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:41.095387 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:41.095591 (+ 204us) client_negotiation.cc:769] Sending connection context
0808 00:42:41.095724 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 00:42:41.095953 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":135,"threads_started":1}
I20260808 00:42:41.096849 30072 heartbeater.cc:347] Connected to a master server at 127.20.105.126:38009
I20260808 00:42:41.097004 30072 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:41.097272 30072 heartbeater.cc:511] Master 127.20.105.126:38009 requested a full tablet report, sending...
I20260808 00:42:41.097836 29408 ts_manager.cc:194] Registered new tserver with Master: 9c03b436f46041eaab3249f0ea8248cf (127.20.105.67:42023)
I20260808 00:42:41.098284 29408 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:40603
I20260808 00:42:41.108541 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/test-admin.keytab.
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:41.129918 20901 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 00:42:41.131987 30080 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:41.131048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.20.105.65:44873)
0808 00:42:41.131247 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:41.131260 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:41.131273 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:42:41.131310 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:41.131312 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:41.131330 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:41.131418 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:41.131861 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60220: BlockingRecv error: recv got EOF from 127.0.0.1:60220 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
W20260808 00:42:41.132071 20901 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc' of type FILE
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:41.133226 20901 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:41.134153 20901 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:41.143564 30080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:41.134825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60232 (local address 127.20.105.65:44873)
0808 00:42:41.134921 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:41.134923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:41.134936 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:42:41.135025 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:41.135026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:41.135039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:41.135100 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:41.135525 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.136491 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.137320 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.137711 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.137780 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:41.140268 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:41.140279 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:41.140279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:41.140288 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:41.142243 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:41.142583 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:41.142586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:41.142587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:41.142648 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:41.142870 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:41.142871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:41.142872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:41.143064 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:41.143207 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:41.143251 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 00:42:41.143345 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 00:42:41.146180 30080 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:41.145512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.20.105.65:44873)
0808 00:42:41.145583 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:41.145586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:41.145631 (+ 45us) server_negotiation.cc:407] Connection header received
0808 00:42:41.145695 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:41.145696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:41.145708 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:41.145759 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:41.146101 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60246: BlockingRecv error: recv got EOF from 127.0.0.1:60246 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 00:42:41.146306 20901 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc)
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:41.147482 20901 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:41.148492 20901 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:41.156937 30080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:41.149108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60256 (local address 127.20.105.65:44873)
0808 00:42:41.149175 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:41.149178 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:41.149242 (+ 64us) server_negotiation.cc:407] Connection header received
0808 00:42:41.149328 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:41.149328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:41.149337 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:41.149398 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:41.149806 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.150371 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.151130 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.151352 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.151441 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:41.153965 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:41.153967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:41.153967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:41.153977 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:41.155514 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:41.155922 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:41.155923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:41.155924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:41.155992 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:41.156319 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:41.156320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:41.156321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:41.156452 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:41.156573 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:41.156706 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 00:42:41.156806 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 00:42:41.159021 30080 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:41.158385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.20.105.65:44873)
0808 00:42:41.158460 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:41.158462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:41.158485 (+ 23us) server_negotiation.cc:407] Connection header received
0808 00:42:41.158583 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:41.158584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:41.158596 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:41.158669 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:41.158958 (+ 289us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60262: BlockingRecv error: recv got EOF from 127.0.0.1:60262 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 00:42:41.159152 20901 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149729697709-20901-0/krb5kdc/krb5cc)
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:41.160459 20901 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:41.161355 20901 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[29345](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:42:41.169313 30080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:41.161956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60272 (local address 127.20.105.65:44873)
0808 00:42:41.162046 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:41.162048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:41.162117 (+ 69us) server_negotiation.cc:407] Connection header received
0808 00:42:41.162169 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:41.162170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:41.162179 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:41.162227 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:41.162643 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.163265 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.164009 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:41.164366 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:41.164445 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:41.166826 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:41.166827 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:41.166827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:41.166834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:41.168234 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:41.168587 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:41.168588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:41.168588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:41.168644 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:41.168828 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:41.168829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:41.168829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:41.168927 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:41.169007 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:41.169108 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 00:42:41.169169 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 00:42:41.170226 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 29498
I20260808 00:42:41.182426 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 29707
I20260808 00:42:41.195326 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 29897
I20260808 00:42:41.215420 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 29364
2026-08-08T00:42:41Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2369 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[30102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[30102](info): set up 2 sockets
Aug 08 00:42:41 dist-test-slave-kvfs krb5kdc[30102](info): commencing operation
krb5kdc: starting...
W20260808 00:42:42.494176 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:42 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.126.keytab.
Entry for principal oryx/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.126.keytab.
I20260808 00:42:42.639598 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44365
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.126.keytab
--principal=oryx/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:44365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:42.735497 30123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:42.735854 30123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:42.735899 30123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:42.737290 30123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:42.737380 30123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:42.737406 30123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:42.737428 30123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:42.737458 30123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:42.739131 30123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:44365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.126.keytab
--principal=oryx/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:42.739475 30123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:42.739768 30123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:42.746904 30134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:42.747077 30129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:42.747009 30123 server_base.cc:1034] running on GCE node
W20260808 00:42:42.747228 30130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:42.747609 30123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:42.748009 30123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:42.749246 30123 hybrid_clock.cc:648] HybridClock initialized: now 1786149762749217 us; error 51 us; skew 500 ppm
Aug 08 00:42:42 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149762, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:42.755532 30123 init.cc:377] Logged in from keytab as oryx/127.20.105.126@KRBTEST.COM (short username oryx)
I20260808 00:42:42.756206 30123 webserver.cc:460] Webserver started at http://127.20.105.126:46547/ using document root <none> and password file <none>
I20260808 00:42:42.756503 30123 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:42.756569 30123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:42.756691 30123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:42.757977 30123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "837aae531f9e489d97bb37425d5df64e"
format_stamp: "Formatted at 2026-08-08 00:42:42 on dist-test-slave-kvfs"
server_key: "7dc7bba346f7c81f8003297bc1007fc9"
server_key_iv: "5560be4f1d4ca984018993e9497d6fcf"
server_key_version: "encryptionkey@0"
I20260808 00:42:42.758359 30123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "837aae531f9e489d97bb37425d5df64e"
format_stamp: "Formatted at 2026-08-08 00:42:42 on dist-test-slave-kvfs"
server_key: "7dc7bba346f7c81f8003297bc1007fc9"
server_key_iv: "5560be4f1d4ca984018993e9497d6fcf"
server_key_version: "encryptionkey@0"
I20260808 00:42:42.759894 30123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 00:42:42.760859 30142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:42.761137 30123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:42.761248 30123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "837aae531f9e489d97bb37425d5df64e"
format_stamp: "Formatted at 2026-08-08 00:42:42 on dist-test-slave-kvfs"
server_key: "7dc7bba346f7c81f8003297bc1007fc9"
server_key_iv: "5560be4f1d4ca984018993e9497d6fcf"
server_key_version: "encryptionkey@0"
I20260808 00:42:42.761323 30123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:42.778038 30123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:42.780366 30123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:42.780519 30123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:42.785410 30123 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:44365
I20260808 00:42:42.785501 30229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:44365 every 8 connection(s)
I20260808 00:42:42.785954 30123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:42.787092 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 30123
I20260808 00:42:42.787199 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:42.787228 30230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:42.787449 20901 external_mini_cluster.cc:1428] Setting key 57ed91896cdde235aa290351eb2a55e3
I20260808 00:42:42.791448 30230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: Bootstrap starting.
I20260808 00:42:42.792338 30230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:42.792734 30230 log.cc:824] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:42.793634 30230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: No bootstrap required, opened a new log
Aug 08 00:42:42 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:42.796483 30230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } }
I20260808 00:42:42.796696 30230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:42.796744 30230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 837aae531f9e489d97bb37425d5df64e, State: Initialized, Role: FOLLOWER
I20260808 00:42:42.796907 30230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } }
I20260808 00:42:42.797022 30230 raft_consensus.cc:388] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:42.797070 30230 raft_consensus.cc:482] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:42.797128 30230 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:42.797986 30230 raft_consensus.cc:504] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } }
I20260808 00:42:42.798120 30230 leader_election.cc:302] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 837aae531f9e489d97bb37425d5df64e; no voters:
I20260808 00:42:42.798300 30230 leader_election.cc:288] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:42.798373 30236 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:42.798563 30236 raft_consensus.cc:686] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 1 LEADER]: Becoming Leader. State: Replica: 837aae531f9e489d97bb37425d5df64e, State: Running, Role: LEADER
I20260808 00:42:42.798662 30230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:42.798703 30236 consensus_queue.cc:237] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } }
I20260808 00:42:42.799291 30238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 837aae531f9e489d97bb37425d5df64e. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } }
I20260808 00:42:42.799398 30238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:42.799292 30237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } }
I20260808 00:42:42.799634 30237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:42.800743 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:42.788203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48940 (local address 127.20.105.126:44365)
0808 00:42:42.788427 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:42.788439 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:42.788478 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:42:42.789299 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:42.789310 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:42.789398 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:42.789587 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:42.790261 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:42.791742 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:42.792627 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:42.793216 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:42.793364 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:42.796407 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:42.796412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:42.796412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:42.796427 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:42.798667 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:42.799116 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:42.799120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:42.799120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:42.799241 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:42.799521 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:42.799522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:42.799522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:42.799915 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:42.800162 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:42.800327 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 00:42:42.800496 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":96,"threads_started":1}
W20260808 00:42:42.800920 30255 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:42:42.800974 30255 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:42:42.801067 30246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:42.801270 30246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:42.802942 30246 catalog_manager.cc:1347] Generated new cluster ID: 432c4cd1754d41b8bbadbafec7495d8d
I20260808 00:42:42.802999 30246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:42.820749 30246 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:42.821406 30246 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:42.827306 30246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: Generated new TSK 0
I20260808 00:42:42.827582 30246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab.
Entry for principal oryx/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab.
I20260808 00:42:42.867954 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44365
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab
--principal=oryx/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:42.997530 30264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:42.997776 30264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:42.997800 30264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:42.999012 30264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:42.999068 30264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:42.999095 30264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:43.000905 30264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab
--principal=oryx/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:43.001211 30264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:43.001502 30264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:43.007280 30271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.007361 30276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.007373 30264 server_base.cc:1034] running on GCE node
W20260808 00:42:43.007339 30272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.007756 30264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:43.008088 30264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:43.009274 30264 hybrid_clock.cc:648] HybridClock initialized: now 1786149763009265 us; error 56 us; skew 500 ppm
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:43.012499 30264 init.cc:377] Logged in from keytab as oryx/127.20.105.65@KRBTEST.COM (short username oryx)
I20260808 00:42:43.012980 30264 webserver.cc:460] Webserver started at http://127.20.105.65:36153/ using document root <none> and password file <none>
I20260808 00:42:43.013198 30264 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:43.013244 30264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:43.013351 30264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:43.014539 30264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "a081ed75d561474bba9a154f25754026"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "43b1d0e56767e737e600a98230487323"
server_key_iv: "61954e3f6b225880a8c2ff9bbc9c8aad"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.014897 30264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "a081ed75d561474bba9a154f25754026"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "43b1d0e56767e737e600a98230487323"
server_key_iv: "61954e3f6b225880a8c2ff9bbc9c8aad"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.016364 30264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:42:43.017158 30286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.017382 30264 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:43.017506 30264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "a081ed75d561474bba9a154f25754026"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "43b1d0e56767e737e600a98230487323"
server_key_iv: "61954e3f6b225880a8c2ff9bbc9c8aad"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.017591 30264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:43.026540 30264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:43.029006 30264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:43.029188 30264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:43.029474 30264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:43.029851 30264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:43.029887 30264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.029914 30264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:43.029961 30264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.037115 30264 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:38339
I20260808 00:42:43.037207 30458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:38339 every 8 connection(s)
I20260808 00:42:43.037599 30264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:43.046675 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 30264
I20260808 00:42:43.046820 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:43.047091 20901 external_mini_cluster.cc:1428] Setting key 699bfacf4d4dcd1dcc2a83a81a625909
I20260808 00:42:43.048830 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.038465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:33533 (local address 127.20.105.126:44365)
0808 00:42:43.038543 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.038545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.038786 (+ 241us) server_negotiation.cc:407] Connection header received
0808 00:42:43.039442 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.039443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.039459 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.039515 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.040694 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.041290 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.042417 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.042663 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.042757 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.045268 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.045270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.045270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.045278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.046983 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.047559 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.047561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.047562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.047645 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.047933 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.047934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.047934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.048143 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.048229 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.048572 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.048719 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:42:43.049057 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.038427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:33533)
0808 00:42:43.038649 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.038692 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.039310 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.039606 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.039610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.040327 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.040580 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.040585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.041397 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.041398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.042295 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.042298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.042769 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.042770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.042919 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.043493 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.043503 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.045140 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.047119 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.047130 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.047133 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.047454 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.047719 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.047720 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.047720 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.047856 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.048217 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.048226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.048408 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.048547 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.048748 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":115,"threads_started":1}
I20260808 00:42:43.049659 30460 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44365
I20260808 00:42:43.049793 30460 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:43.050040 30460 heartbeater.cc:511] Master 127.20.105.126:44365 requested a full tablet report, sending...
I20260808 00:42:43.050750 30170 ts_manager.cc:194] Registered new tserver with Master: a081ed75d561474bba9a154f25754026 (127.20.105.65:38339)
I20260808 00:42:43.051465 30170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:33533
WARNING: no policy specified for oryx/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab.
Entry for principal oryx/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab.
I20260808 00:42:43.083472 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44365
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab
--principal=oryx/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:43.181413 30467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:43.181675 30467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:43.181701 30467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:43.182896 30467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:43.182976 30467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:43.183005 30467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:43.184613 30467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab
--principal=oryx/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:43.184859 30467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:43.185122 30467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:43.191112 30473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.191102 30475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.191159 30479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.191316 30467 server_base.cc:1034] running on GCE node
I20260808 00:42:43.191638 30467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:43.191949 30467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:43.193130 30467 hybrid_clock.cc:648] HybridClock initialized: now 1786149763193109 us; error 44 us; skew 500 ppm
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:43.196239 30467 init.cc:377] Logged in from keytab as oryx/127.20.105.66@KRBTEST.COM (short username oryx)
I20260808 00:42:43.196762 30467 webserver.cc:460] Webserver started at http://127.20.105.66:35243/ using document root <none> and password file <none>
I20260808 00:42:43.197031 30467 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:43.197111 30467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:43.197278 30467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:43.198634 30467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "5be4b9e066454bb3ba060bfb7145e179"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "d774928738c55d4a12b65d1820a6571c"
server_key_iv: "0418b46b3ec632273fbaf48d5c959e56"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.199101 30467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "5be4b9e066454bb3ba060bfb7145e179"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "d774928738c55d4a12b65d1820a6571c"
server_key_iv: "0418b46b3ec632273fbaf48d5c959e56"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.200863 30467 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 00:42:43.201751 30489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.201946 30467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:43.202085 30467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "5be4b9e066454bb3ba060bfb7145e179"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "d774928738c55d4a12b65d1820a6571c"
server_key_iv: "0418b46b3ec632273fbaf48d5c959e56"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.202178 30467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:43.214659 30467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:43.217304 30467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:43.217495 30467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:43.217808 30467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:43.218215 30467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:43.218274 30467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.218325 30467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:43.218356 30467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.226385 30467 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:38391
I20260808 00:42:43.226434 30654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:38391 every 8 connection(s)
I20260808 00:42:43.226884 30467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:43.231423 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 30467
I20260808 00:42:43.231546 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:43.231778 20901 external_mini_cluster.cc:1428] Setting key fd5eb8ad12ef7760389c77320a8c7d36
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:43.241391 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.227919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:49675 (local address 127.20.105.126:44365)
0808 00:42:43.228094 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.228099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.228300 (+ 201us) server_negotiation.cc:407] Connection header received
0808 00:42:43.229049 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.229051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.229069 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.229123 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.230436 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.231180 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.232851 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.233223 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.233337 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.236832 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.236835 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.236835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.236847 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.239135 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.239695 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.239698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.239699 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.239798 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.240331 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.240332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.240333 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.240572 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.240662 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.241094 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.241233 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260808 00:42:43.241576 30657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.227814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:49675)
0808 00:42:43.228127 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.228188 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.228894 (+ 706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.229241 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.229245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.230024 (+ 779us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.230299 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.230304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.231324 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.231326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.232620 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.232625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.233397 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.233399 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.233593 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.234319 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.234335 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.236697 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.239260 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.239269 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.239272 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.239572 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.239913 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.239914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.239914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.240219 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.240689 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.240703 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.240903 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.241014 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.241261 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":119,"threads_started":1}
I20260808 00:42:43.242234 30655 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44365
I20260808 00:42:43.242364 30655 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:43.242681 30655 heartbeater.cc:511] Master 127.20.105.126:44365 requested a full tablet report, sending...
I20260808 00:42:43.243413 30170 ts_manager.cc:194] Registered new tserver with Master: 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391)
I20260808 00:42:43.244226 30170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:49675
WARNING: no policy specified for oryx/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab.
Entry for principal oryx/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab.
I20260808 00:42:43.272490 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44365
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab
--principal=oryx/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:43.370734 30660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:43.370990 30660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:43.371016 30660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:43.372314 30660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:43.372359 30660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:43.372386 30660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:43.373953 30660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab
--principal=oryx/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:43.374275 30660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:43.374584 30660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:43.380601 30672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.380656 30668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.380718 30660 server_base.cc:1034] running on GCE node
W20260808 00:42:43.380604 30669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.381108 30660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:43.381413 30660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:43.382575 30660 hybrid_clock.cc:648] HybridClock initialized: now 1786149763382570 us; error 39 us; skew 500 ppm
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:43.385567 30660 init.cc:377] Logged in from keytab as oryx/127.20.105.67@KRBTEST.COM (short username oryx)
I20260808 00:42:43.386103 30660 webserver.cc:460] Webserver started at http://127.20.105.67:40347/ using document root <none> and password file <none>
I20260808 00:42:43.386351 30660 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:43.386394 30660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:43.386502 30660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:43.387709 30660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "43c44f6b5ab243b59df0aaa532bb86ff"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "919a7743f6c03299387052ff68344f04"
server_key_iv: "5c3a455fc8847063713b8308451c7aa6"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.388139 30660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "43c44f6b5ab243b59df0aaa532bb86ff"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "919a7743f6c03299387052ff68344f04"
server_key_iv: "5c3a455fc8847063713b8308451c7aa6"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.389626 30660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:43.390431 30682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.390645 30660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:43.390719 30660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "43c44f6b5ab243b59df0aaa532bb86ff"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "919a7743f6c03299387052ff68344f04"
server_key_iv: "5c3a455fc8847063713b8308451c7aa6"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.390779 30660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:43.402782 30660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:43.405529 30660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:43.405668 30660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:43.405979 30660 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:43.406339 30660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:43.406373 30660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.406401 30660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:43.406414 30660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.413731 30660 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:40329
I20260808 00:42:43.413808 30847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:40329 every 8 connection(s)
I20260808 00:42:43.414183 30660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:43.420104 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 30660
I20260808 00:42:43.420240 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:43.420464 20901 external_mini_cluster.cc:1428] Setting key bbb05d69dcea18b3125a78d5421e652e
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:43.425024 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.415121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:38027 (local address 127.20.105.126:44365)
0808 00:42:43.415201 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.415219 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.415458 (+ 239us) server_negotiation.cc:407] Connection header received
0808 00:42:43.416191 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.416192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.416208 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.416267 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.417416 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.418002 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.419128 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.419349 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.419433 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.422273 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.422274 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.422274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.422282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.423615 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.424002 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.424003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.424004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.424127 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.424372 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.424373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.424373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.424506 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.424570 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.424846 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.424945 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:42:43.425211 30851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.415088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:38027)
0808 00:42:43.415298 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.415344 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.416010 (+ 666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.416359 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.416363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.417047 (+ 684us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.417287 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.417293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.418110 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.418112 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.419019 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.419022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.419442 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.419444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.419596 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.420319 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.420331 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.422186 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.423688 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.423694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.423696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.423927 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.424193 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.424194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.424194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.424305 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.424570 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.424576 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.424717 (+ 141us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.424808 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.424955 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":96,"threads_started":1}
I20260808 00:42:43.425707 30848 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44365
I20260808 00:42:43.425845 30848 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:43.426075 30848 heartbeater.cc:511] Master 127.20.105.126:44365 requested a full tablet report, sending...
I20260808 00:42:43.426517 30170 ts_manager.cc:194] Registered new tserver with Master: 43c44f6b5ab243b59df0aaa532bb86ff (127.20.105.67:40329)
I20260808 00:42:43.426916 30170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:38027
I20260808 00:42:43.432602 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.105.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.125.keytab.
Entry for principal oryx/127.20.105.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.125.keytab.
I20260808 00:42:43.468076 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.125:43651
--webserver_interface=127.20.105.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.125.keytab
--principal=oryx/127.20.105.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:44365,127.20.105.125:43651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:43.567145 30854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:43.567425 30854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:43.567451 30854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:43.568827 30854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:43.568890 30854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:43.568907 30854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:43.568919 30854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:43.568933 30854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:43.570618 30854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:44365,127.20.105.125:43651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.125.keytab
--principal=oryx/127.20.105.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.125:43651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.105.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
W20260808 00:42:43.570793 30854 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.105.126:44365,127.20.105.125:43651'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 00:42:43.570986 30854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:43.571287 30854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:43.577122 30864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.577222 30866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.577121 30862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.577545 30854 server_base.cc:1034] running on GCE node
I20260808 00:42:43.577883 30854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:43.578299 30854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:43.579473 30854 hybrid_clock.cc:648] HybridClock initialized: now 1786149763579448 us; error 51 us; skew 500 ppm
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:43.582624 30854 init.cc:377] Logged in from keytab as oryx/127.20.105.125@KRBTEST.COM (short username oryx)
I20260808 00:42:43.583205 30854 webserver.cc:460] Webserver started at http://127.20.105.125:38061/ using document root <none> and password file <none>
I20260808 00:42:43.583482 30854 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:43.583559 30854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:43.583732 30854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:43.585119 30854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data/instance:
uuid: "707d92be9cbc45278e303eff39258662"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "0b6bcdd84b6f1bb63cdcd171b131e203"
server_key_iv: "c8bda02a21c8911cd61a0c40d144ee74"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.585552 30854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal/instance:
uuid: "707d92be9cbc45278e303eff39258662"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "0b6bcdd84b6f1bb63cdcd171b131e203"
server_key_iv: "c8bda02a21c8911cd61a0c40d144ee74"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.587447 30854 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 00:42:43.588418 30880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.588647 30854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:43.588775 30854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
uuid: "707d92be9cbc45278e303eff39258662"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "0b6bcdd84b6f1bb63cdcd171b131e203"
server_key_iv: "c8bda02a21c8911cd61a0c40d144ee74"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.588866 30854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:43.609151 30854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:43.612011 30854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:43.612248 30854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:43.618209 30854 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.125:43651
I20260808 00:42:43.618266 30953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.125:43651 every 8 connection(s)
I20260808 00:42:43.618734 30854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
I20260808 00:42:43.620172 30954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:43.622447 30954 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:42:43.626529 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 30854
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.125@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.105.125@KRBTEST.COM
I20260808 00:42:43.634610 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.623374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48948 (local address 127.20.105.126:44365)
0808 00:42:43.623465 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.623486 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.623733 (+ 247us) server_negotiation.cc:407] Connection header received
0808 00:42:43.624423 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.624425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.624442 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.624503 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.625291 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.625900 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.627532 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.627906 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.627969 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.630971 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.630974 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.630974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.630985 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.632615 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.633125 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.633127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.633127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.633203 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.633534 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.633535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.633535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.633839 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.633954 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.634392 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.634499 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:42:43.634835 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.623338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:48948)
0808 00:42:43.623579 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.623626 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.624283 (+ 657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.624596 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.624600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.624926 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.625183 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.625188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.626020 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.626021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.627406 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.627410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.628003 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.628005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.628217 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.629006 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.629017 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.630859 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.632724 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.632734 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.632736 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.633032 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.633292 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.633293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.633294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.633460 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.633984 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.633993 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.634202 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.634335 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.634527 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":112,"threads_started":1}
I20260808 00:42:43.636096 30954 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:42:43.637673 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.627282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.20.105.125:43651)
0808 00:42:43.627570 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.627586 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.627609 (+ 23us) server_negotiation.cc:407] Connection header received
0808 00:42:43.627633 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.627635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.627657 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.627755 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.628736 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.629536 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.630499 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.630865 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.630979 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.634008 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.634018 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.634018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.634026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.636142 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.636605 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.636609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.636610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.636695 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.636997 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.636998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.636998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.637163 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.637389 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.637440 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.637532 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":141,"threads_started":1}
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.125@KRBTEST.COM for oryx/127.20.105.125@KRBTEST.COM
I20260808 00:42:43.644205 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.636452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.0.0.1:47246)
0808 00:42:43.636526 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.636537 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.636577 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.636901 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.636902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.637049 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.637104 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.637106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.637920 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.637921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.638579 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.638580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.638959 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.638959 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.639019 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.639387 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.639392 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.640766 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.643202 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.643203 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.643204 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.643386 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.643591 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.643591 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.643592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.643648 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.643884 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.643885 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.643922 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.643975 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.644128 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 00:42:43.644433 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.636522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47246 (local address 127.20.105.125:43651)
0808 00:42:43.636756 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.636758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.636764 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:42:43.636781 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.636782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.636794 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.636898 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.637187 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.637836 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.638652 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.638889 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.638949 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.640847 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.640849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.640849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.640856 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.643089 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.643460 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.643461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.643462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.643521 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.643707 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.643707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.643707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.643826 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.643886 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.644146 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.644268 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":152,"threads_started":1}
I20260808 00:42:43.646193 30954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Bootstrap starting.
I20260808 00:42:43.647048 30954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:43.647405 30954 log.cc:824] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:43.648504 30954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: No bootstrap required, opened a new log
I20260808 00:42:43.650214 30954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } }
I20260808 00:42:43.650398 30954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:43.650453 30954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707d92be9cbc45278e303eff39258662, State: Initialized, Role: FOLLOWER
I20260808 00:42:43.650614 30954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } }
I20260808 00:42:43.651003 30954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:43.650930 30963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } } }
I20260808 00:42:43.651106 30963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:42:43.653335 30978 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:42:43.653414 30978 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:42:43.655790 30854 master_runner.cc:428] Detected that this master 707d92be9cbc45278e303eff39258662 is joining an existing cluster
I20260808 00:42:43.655871 30854 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.105.125:43651
I20260808 00:42:43.665794 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.657482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.20.105.126:44365)
0808 00:42:43.657546 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.657549 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.657733 (+ 184us) server_negotiation.cc:407] Connection header received
0808 00:42:43.657862 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.657864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.657881 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.657941 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.658431 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.659253 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.660875 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.661193 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.661268 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.662227 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.662228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.662229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.662238 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.664145 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.664751 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.664752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.664752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.664828 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.665110 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.665111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.665111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.665297 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.665351 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.665570 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.665669 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:42:43.665959 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.657645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47260 (local address 127.20.105.125:43651)
0808 00:42:43.657687 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.657689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.657753 (+ 64us) server_negotiation.cc:407] Connection header received
0808 00:42:43.657808 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.657808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.657820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.657882 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.658443 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.659037 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.660361 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.660729 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.660808 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.662634 (+ 1826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.662636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.662636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.662644 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.664578 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.664977 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.664978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.664979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.665030 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.665281 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.665282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.665282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.665424 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.665493 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.665670 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.665801 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":21}
I20260808 00:42:43.666169 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.657425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:48960)
0808 00:42:43.657647 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.657662 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.657743 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.658010 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.658011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.658206 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.658332 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.658335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.659342 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.659343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.660262 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.660264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.661283 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.661285 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.661359 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.661755 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.661759 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.662106 (+ 347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.664243 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.664245 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.664246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.664633 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.664903 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.664904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.664904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.665032 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.665399 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.665400 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.665441 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.665533 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.666058 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"spinlock_wait_cycles":12160,"thread_start_us":148,"threads_started":1}
I20260808 00:42:43.666654 30987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.657430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.0.0.1:47260)
0808 00:42:43.657647 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.657662 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.657743 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.657886 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.657887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.658204 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.658332 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.658335 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.659140 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.659141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.660254 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.660257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.660842 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.660843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.660912 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.661554 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.661560 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.662542 (+ 982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.664669 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.664671 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.664672 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.664898 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.665114 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.665114 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.665115 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.665206 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.665499 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.665500 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.665551 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.665700 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.666536 (+ 836us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":83,"thread_start_us":104,"threads_started":1}
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149762, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.126@KRBTEST.COM for oryx/127.20.105.125@KRBTEST.COM
I20260808 00:42:43.678368 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.668979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.0.0.1:47272)
0808 00:42:43.669228 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.669301 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.669393 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.669546 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.669550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.669955 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.670037 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.670040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.670674 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.670675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.671707 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.671710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.672067 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.672068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.672172 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.672720 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.672726 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.674958 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.677000 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.677008 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.677011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.677367 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.677603 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.677603 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.677604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.677675 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.677957 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.677965 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.678020 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.678124 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.678256 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":124,"threads_started":1}
I20260808 00:42:43.678380 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.669082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47272 (local address 127.20.105.125:43651)
0808 00:42:43.669143 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.669145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.669380 (+ 235us) server_negotiation.cc:407] Connection header received
0808 00:42:43.669475 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.669476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.669486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.669529 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.670113 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.670591 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.671791 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.671969 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.672066 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.675097 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.675100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.675100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.675111 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.676906 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.677448 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.677450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.677450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.677524 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.677743 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.677744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.677744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.677890 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.677962 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.678191 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.678279 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:42:43.679291 30170 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.105.125:43651: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "707d92be9cbc45278e303eff39258662"
member_type: NON_VOTER
last_known_addr {
host: "127.20.105.125"
port: 43651
}
attrs {
promote: true
}
}
dest_uuid: "837aae531f9e489d97bb37425d5df64e"
cas_config_opid_index: -1
I20260808 00:42:43.679746 30170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } }
I20260808 00:42:43.680460 30994 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 707d92be9cbc45278e303eff39258662 (127.20.105.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } }
I20260808 00:42:43.680869 30914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:43.681674 30994 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.105.125:43651
I20260808 00:42:43.682132 30854 master.cc:537] Master@127.20.105.125:43651 shutting down...
I20260808 00:42:43.682134 30992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } }
I20260808 00:42:43.682228 30992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:43.682482 30914 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Refusing update from remote peer 837aae531f9e489d97bb37425d5df64e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 00:42:43.682874 30992 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Connected to new peer: Peer: permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:42:43.683892 30143 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 00:42:43.684741 30143 consensus_peers.cc:487] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e -> Peer 707d92be9cbc45278e303eff39258662 (127.20.105.125:43651): Couldn't send request to peer 707d92be9cbc45278e303eff39258662. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 00:42:43.686076 30854 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:43.686188 30854 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:43.686249 30854 tablet_replica.cc:330] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: stopping tablet replica
I20260808 00:42:43.699747 30854 master.cc:552] Master@127.20.105.125:43651 shutdown complete.
I20260808 00:42:43.699900 30854 master_runner.cc:305] Clearing existing system tablet
I20260808 00:42:43.700124 30854 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:43.700198 30854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:43.700984 30854 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:43.701602 31001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.701781 30854 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:43.701838 30854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:43.701862 30854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
uuid: "707d92be9cbc45278e303eff39258662"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "0b6bcdd84b6f1bb63cdcd171b131e203"
server_key_iv: "c8bda02a21c8911cd61a0c40d144ee74"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.702955 30854 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:43.704845 30854 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 00:42:43.704932 30854 log.cc:1196] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 00:42:43.705158 30854 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Deleting consensus metadata
I20260808 00:42:43.705731 30854 master_runner.cc:315] Copying system tablet from 127.20.105.126:44365
I20260808 00:42:43.706455 30854 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: tablet copy: Beginning tablet copy session from remote peer at address 127.20.105.126:44365
I20260808 00:42:43.713829 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.706950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48966 (local address 127.20.105.126:44365)
0808 00:42:43.707031 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.707034 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.707136 (+ 102us) server_negotiation.cc:407] Connection header received
0808 00:42:43.707215 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.707216 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.707228 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.707284 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.707762 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.708474 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.709631 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.709860 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.709919 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.711179 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.711181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.711181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.711187 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.712719 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.713102 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.713103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.713103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.713160 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.713367 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.713368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.713368 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.713478 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.713530 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.713686 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.713751 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:42:43.713893 31011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.706892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:48966)
0808 00:42:43.707039 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:43.707054 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:43.707141 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:43.707319 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:43.707320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:43.707592 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:43.707688 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.707691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.708554 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.708556 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.709516 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:43.709519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.709945 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.709946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:43.710023 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.710611 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:43.710617 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:43.711085 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:43.712802 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.712804 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.712804 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.713027 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.713226 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:43.713227 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:43.713228 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:43.713308 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:43.713538 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:43.713539 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:43.713588 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 00:42:43.713659 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:42:43.713772 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260808 00:42:43.714627 30204 tablet_copy_service.cc:140] P 837aae531f9e489d97bb37425d5df64e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 707d92be9cbc45278e303eff39258662 ({username='oryx', principal='oryx/127.20.105.125@KRBTEST.COM'} at 127.0.0.1:48966)
I20260808 00:42:43.714776 30204 tablet_copy_service.cc:161] P 837aae531f9e489d97bb37425d5df64e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 707d92be9cbc45278e303eff39258662 at {username='oryx', principal='oryx/127.20.105.125@KRBTEST.COM'} at 127.0.0.1:48966: session id = 707d92be9cbc45278e303eff39258662-00000000000000000000000000000000
I20260808 00:42:43.715871 30204 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 00:42:43.716686 30854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 00:42:43.718122 30854 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: tablet copy: Starting download of 0 data blocks...
I20260808 00:42:43.718297 30854 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: tablet copy: Starting download of 1 WAL segments...
I20260808 00:42:43.719344 30854 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 00:42:43.720345 30204 tablet_copy_service.cc:342] P 837aae531f9e489d97bb37425d5df64e: Request end of tablet copy session 707d92be9cbc45278e303eff39258662-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.105.125@KRBTEST.COM'} at 127.0.0.1:48966
I20260808 00:42:43.720414 30204 tablet_copy_service.cc:433] P 837aae531f9e489d97bb37425d5df64e: ending tablet copy session 707d92be9cbc45278e303eff39258662-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 707d92be9cbc45278e303eff39258662
I20260808 00:42:43.723393 30854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:43.724848 31015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.724882 31013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.725076 30854 server_base.cc:1034] running on GCE node
W20260808 00:42:43.724893 31012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.725255 30854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:43.725481 30854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:43.726637 30854 hybrid_clock.cc:648] HybridClock initialized: now 1786149763726626 us; error 31 us; skew 500 ppm
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:43.728240 30854 init.cc:377] Logged in from keytab as oryx/127.20.105.125@KRBTEST.COM (short username oryx)
I20260808 00:42:43.728628 30854 webserver.cc:460] Webserver started at http://127.20.105.125:45669/ using document root <none> and password file <none>
I20260808 00:42:43.728785 30854 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:43.728826 30854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:43.729524 30854 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:43.730160 31026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.730283 30854 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:43.730334 30854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
uuid: "707d92be9cbc45278e303eff39258662"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "0b6bcdd84b6f1bb63cdcd171b131e203"
server_key_iv: "c8bda02a21c8911cd61a0c40d144ee74"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.730412 30854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:43.736790 30854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:43.738873 30854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:43.744846 30854 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.125:43651
I20260808 00:42:43.745251 31110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.125:43651 every 8 connection(s)
I20260808 00:42:43.745383 30854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
I20260808 00:42:43.746666 31112 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:42:43.747386 31112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Bootstrap starting.
I20260808 00:42:43.751659 31112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:42:43.751979 31112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Bootstrap complete.
I20260808 00:42:43.752355 31112 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } }
I20260808 00:42:43.752496 31112 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 707d92be9cbc45278e303eff39258662, State: Initialized, Role: LEARNER
I20260808 00:42:43.752552 31112 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } }
I20260808 00:42:43.752785 31112 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:43.752774 31115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } }
I20260808 00:42:43.752894 31115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:42:43.754873 31128 catalog_manager.cc:1259] Loaded cluster ID: 432c4cd1754d41b8bbadbafec7495d8d
I20260808 00:42:43.755676 31128 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: loading cluster ID for follower catalog manager: success
I20260808 00:42:43.756590 31128 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: acquiring CA information for follower catalog manager: success
I20260808 00:42:43.756904 31128 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:42:43.798233 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:43.790159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47310 (local address 127.20.105.125:43651)
0808 00:42:43.790389 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:43.790391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:43.790399 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:43.790432 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:43.790433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:43.790444 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:43.790538 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:43.791211 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.792007 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.793307 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:43.793527 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:43.793595 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:43.795189 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:43.795191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:43.795191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:43.795198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:43.796879 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.797272 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.797274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.797274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.797347 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:43.797578 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:43.797579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:43.797580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:43.797708 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:43.797822 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:43.797947 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 00:42:43.798058 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for oryx/127.20.105.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.124.keytab.
Entry for principal oryx/127.20.105.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.124.keytab.
I20260808 00:42:43.837431 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.124:36103
--webserver_interface=127.20.105.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.124.keytab
--principal=oryx/127.20.105.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:44365,127.20.105.125:43651,127.20.105.124:36103
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:43.949651 31132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:43.949945 31132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:43.949975 31132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:43.951345 31132 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:43.951406 31132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:43.951422 31132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:43.951436 31132 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:43.951448 31132 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:43.953362 31132 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:44365,127.20.105.125:43651,127.20.105.124:36103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.124.keytab
--principal=oryx/127.20.105.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.124:36103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.105.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:43.953713 31132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:43.954022 31132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:43.960101 31139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.960101 31142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:43.960112 31145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:43.960480 31132 server_base.cc:1034] running on GCE node
I20260808 00:42:43.960944 31132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:43.961334 31132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:43.962513 31132 hybrid_clock.cc:648] HybridClock initialized: now 1786149763962492 us; error 42 us; skew 500 ppm
Aug 08 00:42:43 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:43.965788 31132 init.cc:377] Logged in from keytab as oryx/127.20.105.124@KRBTEST.COM (short username oryx)
I20260808 00:42:43.966382 31132 webserver.cc:460] Webserver started at http://127.20.105.124:42431/ using document root <none> and password file <none>
I20260808 00:42:43.966617 31132 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:43.966662 31132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:43.966774 31132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:43.968120 31132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data/instance:
uuid: "700e8c77909849b49ea4c79ccd7ac037"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "09717d115cd4bb20e35faab546892e51"
server_key_iv: "735a4cfcdc74f7fbe9539f86360c8b13"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.968513 31132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal/instance:
uuid: "700e8c77909849b49ea4c79ccd7ac037"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "09717d115cd4bb20e35faab546892e51"
server_key_iv: "735a4cfcdc74f7fbe9539f86360c8b13"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.970119 31132 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:42:43.970944 31153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:43.971174 31132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:43.971283 31132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
uuid: "700e8c77909849b49ea4c79ccd7ac037"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "09717d115cd4bb20e35faab546892e51"
server_key_iv: "735a4cfcdc74f7fbe9539f86360c8b13"
server_key_version: "encryptionkey@0"
I20260808 00:42:43.971354 31132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:43.994321 31132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:43.997213 31132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:43.997457 31132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:44.004496 31132 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.124:36103
I20260808 00:42:44.004545 31225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.124:36103 every 8 connection(s)
I20260808 00:42:44.005034 31132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
I20260808 00:42:44.006476 31228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:44.007087 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 31132
I20260808 00:42:44.008941 31228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.105.124@KRBTEST.COM
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.124@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:44.020869 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.009870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.20.105.126:44365)
0808 00:42:44.009932 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.009934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.009996 (+ 62us) server_negotiation.cc:407] Connection header received
0808 00:42:44.010088 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.010092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.010108 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.010155 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.011167 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.011808 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.013429 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.013773 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.013834 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.017257 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.017278 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.017279 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.017289 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.019006 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.019770 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.019772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.019772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.019845 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.020167 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.020168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.020168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.020366 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.020455 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.020653 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.020750 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:42:44.021016 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.009687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:48978)
0808 00:42:44.009864 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.009913 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.009994 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.010190 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.010193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.010702 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.011031 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.011039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.012248 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.012249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.013316 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.013319 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.014003 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.014005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.014099 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.014613 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.014623 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.017112 (+ 2489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.019385 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.019392 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.019394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.019663 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.019931 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.019932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.019932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.019994 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.020453 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.020462 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.020520 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.020608 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.020775 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":113,"threads_started":1}
I20260808 00:42:44.021059 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.008002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.20.105.124:36103)
0808 00:42:44.008276 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.008286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.008321 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:42:44.008913 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.008923 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.008992 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.009192 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.010065 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.011613 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.012561 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.013222 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.013357 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.016669 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.016686 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.016687 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.016694 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.019005 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.019380 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.019384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.019384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.019493 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.019719 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.019720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.019720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.020073 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.020342 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.020429 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.020620 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
I20260808 00:42:44.021783 31228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.124@KRBTEST.COM for oryx/127.20.105.125@KRBTEST.COM
I20260808 00:42:44.029577 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.022117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.0.0.1:47324)
0808 00:42:44.022177 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.022188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.022247 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.022427 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.022428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.022589 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.022632 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.022634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.023390 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.023391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.023934 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.023936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.024394 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.024395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.024457 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.024952 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.024957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.026375 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.028293 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.028294 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.028294 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.028513 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.028745 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.028746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.028746 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.028822 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.029223 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.029225 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.029276 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.029347 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.029473 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 00:42:44.029606 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.022213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47324 (local address 127.20.105.125:43651)
0808 00:42:44.022288 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.022291 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.022302 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:44.022335 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.022336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.022352 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.022434 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.022722 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.023290 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.024060 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.024301 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.024398 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.026489 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.026491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.026492 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.026500 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.028206 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.028598 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.028599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.028600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.028661 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.028971 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.028972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.028973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.029124 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.029200 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.029413 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.029516 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:42:44.030189 31228 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149763, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.124@KRBTEST.COM for oryx/127.20.105.124@KRBTEST.COM
I20260808 00:42:44.037917 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.030706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.20.105.124:36103)
0808 00:42:44.030748 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.030750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.030764 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:42:44.030782 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.030783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.030791 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.030842 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.031163 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.031717 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.032529 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.032746 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.032802 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.034972 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.034974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.034974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.034982 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.036513 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.036944 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.036945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.036945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.037016 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.037336 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.037337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.037337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.037486 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.037634 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.037730 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.037826 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:42:44.038028 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.030547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:36103 (local address 127.0.0.1:50926)
0808 00:42:44.030625 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.030638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.030689 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.030857 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.030859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.031034 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.031078 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.031080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.031809 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.031810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.032450 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.032453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.032817 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.032818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.032865 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.033268 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.033273 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.034826 (+ 1553us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.036614 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.036615 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.036615 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.036794 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.037157 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.037159 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.037159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.037233 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.037565 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.037566 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.037616 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.037698 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.037912 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":95}
I20260808 00:42:44.040285 31228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Bootstrap starting.
I20260808 00:42:44.041296 31228 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:44.041646 31228 log.cc:824] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:44.042764 31228 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: No bootstrap required, opened a new log
I20260808 00:42:44.045642 31228 raft_consensus.cc:348] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } }
I20260808 00:42:44.045840 31228 raft_consensus.cc:374] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:44.045868 31228 raft_consensus.cc:729] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 700e8c77909849b49ea4c79ccd7ac037, State: Initialized, Role: FOLLOWER
I20260808 00:42:44.046047 31228 consensus_queue.cc:260] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } }
I20260808 00:42:44.046463 31228 sys_catalog.cc:564] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:44.046470 31233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } } }
I20260808 00:42:44.046674 31233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:42:44.049371 31246 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:42:44.049456 31246 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:42:44.052608 31132 master_runner.cc:428] Detected that this master 700e8c77909849b49ea4c79ccd7ac037 is joining an existing cluster
I20260808 00:42:44.052703 31132 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.105.124:36103
I20260808 00:42:44.052709 30460 heartbeater.cc:503] Master 127.20.105.126:44365 was elected leader, sending a full tablet report...
I20260808 00:42:44.063688 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.054546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.20.105.126:44365)
0808 00:42:44.054611 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.054614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.055053 (+ 439us) server_negotiation.cc:407] Connection header received
0808 00:42:44.055179 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.055181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.055196 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.055233 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.055957 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.056851 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.058099 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.058424 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.058504 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.060497 (+ 1993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.060498 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.060499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.060510 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.062253 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.062734 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.062736 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.062736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.062809 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.063045 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.063045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.063046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.063190 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.063249 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.063510 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.063602 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:42:44.063792 31259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.054497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:48980)
0808 00:42:44.054961 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.054972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.055087 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.055318 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.055320 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.055755 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.055849 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.055853 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.056960 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.056961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.057988 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.057990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.058705 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.058706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.058824 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.059654 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.059660 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.060390 (+ 730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.062350 (+ 1960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.062352 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.062352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.062635 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.062887 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.062888 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.062888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.062975 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.063274 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.063276 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.063357 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.063456 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.063639 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":580,"threads_started":1}
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149762, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.126@KRBTEST.COM for oryx/127.20.105.124@KRBTEST.COM
I20260808 00:42:44.073280 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 30264
I20260808 00:42:44.079083 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.066754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:36103 (local address 127.0.0.1:50950)
0808 00:42:44.066834 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.066846 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.066899 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.067377 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.067379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.067611 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.067680 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.067682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.068727 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.068728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.069742 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.069747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.070312 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.070313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.070399 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.070900 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.070907 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.073648 (+ 2741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.077080 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.077082 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.077083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.077391 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.077661 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.077662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.077663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.077748 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.078509 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.078510 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.078592 (+ 82us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.078658 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.078849 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 00:42:44.080248 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.066863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50950 (local address 127.20.105.124:36103)
0808 00:42:44.067194 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.067196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.067204 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:44.067267 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.067267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.067277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.067341 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.067803 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.068638 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.069870 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.070212 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.070295 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.074141 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.074143 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.074143 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.074151 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.076941 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.077498 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.077499 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.077500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.077573 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.077820 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.077821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.077821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.078403 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.079798 (+ 1395us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.079888 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.080043 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":152,"threads_started":1}
I20260808 00:42:44.093684 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.054666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47330 (local address 127.20.105.125:43651)
0808 00:42:44.054713 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.054715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.055202 (+ 487us) server_negotiation.cc:407] Connection header received
0808 00:42:44.055304 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.055305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.055318 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.055354 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.056188 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.057196 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.058430 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.058804 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.058854 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.073526 (+ 14672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.073530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.073530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.073543 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.091669 (+ 18126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.092582 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.092585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.092586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.092660 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.092868 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.092869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.092869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.093060 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.093119 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.093354 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.093497 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:42:44.094667 31262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.054714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.0.0.1:47330)
0808 00:42:44.055135 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.055145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.055228 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.055429 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.055429 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.055937 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.056082 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.056085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.057302 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.057303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.058321 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.058323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.058950 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.058951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.059033 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.072827 (+ 13794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.072833 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.073358 (+ 525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.091803 (+ 18445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.091805 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.091806 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.092490 (+ 684us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.092728 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.092729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.092729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.092800 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.093157 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.093159 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.093220 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.094277 (+ 1057us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.094458 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":172,"threads_started":1}
I20260808 00:42:44.095723 30170 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.105.124:36103: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037"
member_type: NON_VOTER
last_known_addr {
host: "127.20.105.124"
port: 36103
}
attrs {
promote: true
}
}
dest_uuid: "837aae531f9e489d97bb37425d5df64e"
cas_config_opid_index: 5
I20260808 00:42:44.096261 30170 consensus_queue.cc:237] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } }
I20260808 00:42:44.096529 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.055273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:36103 (local address 127.0.0.1:50940)
0808 00:42:44.055806 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.055817 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.055915 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.056160 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.056161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.056536 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.056777 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.056780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.057734 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.057735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.058874 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.058877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.059314 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.059314 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.059392 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.072648 (+ 13256us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.072654 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.090009 (+ 17355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.094821 (+ 4812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.094823 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.094823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.095065 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.095295 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.095296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.095296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.095355 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.095766 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.095767 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.095814 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.095879 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.096011 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":331,"threads_started":1}
I20260808 00:42:44.096933 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.055674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.20.105.124:36103)
0808 00:42:44.055724 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.055726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.055888 (+ 162us) server_negotiation.cc:407] Connection header received
0808 00:42:44.055988 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.055990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.056002 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.056096 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.056877 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.057645 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.058966 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.059225 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.059258 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.090317 (+ 31059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.090319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.090319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.090329 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.092229 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.095168 (+ 2939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.095169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.095170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.095231 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.095413 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.095414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.095414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.095650 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.095740 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.096758 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.096852 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:42:44.097044 30992 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.098789 30992 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.105.124:36103
I20260808 00:42:44.098896 30992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.098978 30992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:44.099443 31132 master.cc:537] Master@127.20.105.124:36103 shutting down...
I20260808 00:42:44.102432 31189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:44.103583 31189 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Refusing update from remote peer 837aae531f9e489d97bb37425d5df64e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 00:42:44.104684 31266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Connected to new peer: Peer: permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:44.105365 31132 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:44.105481 31132 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:44.105537 31132 tablet_replica.cc:330] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: stopping tablet replica
W20260808 00:42:44.105760 30152 consensus_peers.cc:487] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e -> Peer 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124:36103): Couldn't send request to peer 700e8c77909849b49ea4c79ccd7ac037. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 00:42:44.107551 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.097443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.20.105.125:43651)
0808 00:42:44.103959 (+ 6516us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.103962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.103976 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:42:44.104011 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.104013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.104069 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.104150 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:44.104413 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.105182 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.106558 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.107113 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.107216 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.107318 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.107414 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.107470 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6472}
I20260808 00:42:44.107549 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.097224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.0.0.1:47336)
0808 00:42:44.097269 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.097308 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.097354 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.104253 (+ 6899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.104255 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.104258 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:44.104330 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.104332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.105301 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.105303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.106449 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.106453 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.107215 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.107216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.107281 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.107314 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.107375 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.107435 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 00:42:44.107920 31066 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 LEARNER]: Refusing update from remote peer 837aae531f9e489d97bb37425d5df64e: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 00:42:44.108208 31266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Connected to new peer: Peer: permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 00:42:44.109009 31066 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.110271 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:38339
--local_ip_for_outbound_sockets=127.20.105.65
--tserver_master_addrs=127.20.105.126:44365
--webserver_port=36153
--webserver_interface=127.20.105.65
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab
--principal=oryx/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651,127.20.105.124:36103 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:42:44.110865 30994 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: attempting to promote NON_VOTER 707d92be9cbc45278e303eff39258662 to VOTER
I20260808 00:42:44.111145 30994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } }
W20260808 00:42:44.112167 30152 consensus_peers.cc:487] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e -> Peer 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124:36103): Couldn't send request to peer 700e8c77909849b49ea4c79ccd7ac037. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 00:42:44.112402 31066 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 LEARNER]: Refusing update from remote peer 837aae531f9e489d97bb37425d5df64e: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 00:42:44.112591 31266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Connected to new peer: Peer: permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 00:42:44.113238 30992 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 707d92be9cbc45278e303eff39258662 (127.20.105.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.113607 31115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 837aae531f9e489d97bb37425d5df64e. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.113751 31115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:42:44.113803 31115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: NON_VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: true } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.113874 31115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:42:44.114614 30992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.114703 30992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:44.115633 30994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.115725 30994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:44.116194 31066 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 707d92be9cbc45278e303eff39258662 (127.20.105.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.117489 31269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.117766 31269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:42:44.127310 31132 master.cc:552] Master@127.20.105.124:36103 shutdown complete.
I20260808 00:42:44.127399 31132 master_runner.cc:305] Clearing existing system tablet
I20260808 00:42:44.127595 31132 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:44.127655 31132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:44.128715 31132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:44.129326 31275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.129459 31132 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:44.129510 31132 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:42:44.129534 31132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
uuid: "700e8c77909849b49ea4c79ccd7ac037"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "09717d115cd4bb20e35faab546892e51"
server_key_iv: "735a4cfcdc74f7fbe9539f86360c8b13"
server_key_version: "encryptionkey@0"
I20260808 00:42:44.130635 31132 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:44.132535 31132 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 00:42:44.132627 31132 log.cc:1196] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 00:42:44.132874 31132 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Deleting consensus metadata
I20260808 00:42:44.133033 31132 master_runner.cc:315] Copying system tablet from 127.20.105.126:44365
I20260808 00:42:44.133881 31132 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: tablet copy: Beginning tablet copy session from remote peer at address 127.20.105.126:44365
I20260808 00:42:44.142515 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.134410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.20.105.126:44365)
0808 00:42:44.134481 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.134484 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.134653 (+ 169us) server_negotiation.cc:407] Connection header received
0808 00:42:44.134724 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.134725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.134741 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.134812 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.135291 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.136159 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.137329 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.137672 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.137755 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.139209 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.139210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.139211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.139220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.141099 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.141540 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.141541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.141542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.141616 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.141852 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.141853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.141853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.142036 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.142117 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.142291 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.142389 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:42:44.142648 31283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.134316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:48996)
0808 00:42:44.134540 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.134553 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.134649 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.134843 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.134844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.135105 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.135201 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.135204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.136269 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.136270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.137222 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.137226 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.137849 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.137850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.137932 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.138607 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.138614 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.139103 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.141192 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.141194 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.141195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.141445 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.141694 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.141695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.141695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.141784 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.142113 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.142114 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.142170 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.142247 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.142380 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":80,"threads_started":1}
I20260808 00:42:44.143364 30204 tablet_copy_service.cc:140] P 837aae531f9e489d97bb37425d5df64e: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 700e8c77909849b49ea4c79ccd7ac037 ({username='oryx', principal='oryx/127.20.105.124@KRBTEST.COM'} at 127.0.0.1:48996)
I20260808 00:42:44.143443 30204 tablet_copy_service.cc:161] P 837aae531f9e489d97bb37425d5df64e: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 700e8c77909849b49ea4c79ccd7ac037 at {username='oryx', principal='oryx/127.20.105.124@KRBTEST.COM'} at 127.0.0.1:48996: session id = 700e8c77909849b49ea4c79ccd7ac037-00000000000000000000000000000000
I20260808 00:42:44.144236 30204 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 00:42:44.144842 31132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 00:42:44.146488 31132 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: tablet copy: Starting download of 0 data blocks...
I20260808 00:42:44.146677 31132 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: tablet copy: Starting download of 1 WAL segments...
I20260808 00:42:44.147636 31132 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 00:42:44.148854 30204 tablet_copy_service.cc:342] P 837aae531f9e489d97bb37425d5df64e: Request end of tablet copy session 700e8c77909849b49ea4c79ccd7ac037-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.105.124@KRBTEST.COM'} at 127.0.0.1:48996
I20260808 00:42:44.148924 30204 tablet_copy_service.cc:433] P 837aae531f9e489d97bb37425d5df64e: ending tablet copy session 700e8c77909849b49ea4c79ccd7ac037-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 700e8c77909849b49ea4c79ccd7ac037
I20260808 00:42:44.153554 31132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:44.155189 31284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:44.155249 31287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:44.155483 31285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:44.155535 31132 server_base.cc:1034] running on GCE node
I20260808 00:42:44.155939 31132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:44.156177 31132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:44.157320 31132 hybrid_clock.cc:648] HybridClock initialized: now 1786149764157337 us; error 56 us; skew 500 ppm
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:44.159210 31132 init.cc:377] Logged in from keytab as oryx/127.20.105.124@KRBTEST.COM (short username oryx)
I20260808 00:42:44.159579 31132 webserver.cc:460] Webserver started at http://127.20.105.124:46513/ using document root <none> and password file <none>
I20260808 00:42:44.159716 31132 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:44.159754 31132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:44.160655 31132 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:44.161394 31298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.161602 31132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:44.161657 31132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
uuid: "700e8c77909849b49ea4c79ccd7ac037"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "09717d115cd4bb20e35faab546892e51"
server_key_iv: "735a4cfcdc74f7fbe9539f86360c8b13"
server_key_version: "encryptionkey@0"
I20260808 00:42:44.161736 31132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:44.188161 31132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:44.190238 31132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:44.194151 31132 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.124:36103
I20260808 00:42:44.194682 31132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
I20260808 00:42:44.194697 31366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.124:36103 every 8 connection(s)
I20260808 00:42:44.195851 31368 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:42:44.196465 31368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Bootstrap starting.
I20260808 00:42:44.200151 31368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:42:44.200414 31368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: Bootstrap complete.
I20260808 00:42:44.200706 31368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } }
I20260808 00:42:44.200825 31368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 700e8c77909849b49ea4c79ccd7ac037, State: Initialized, Role: LEARNER
I20260808 00:42:44.200867 31368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } }
I20260808 00:42:44.201102 31368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:44.203125 31372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.203249 31372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:42:44.203500 31388 catalog_manager.cc:1259] Loaded cluster ID: 432c4cd1754d41b8bbadbafec7495d8d
I20260808 00:42:44.203533 31388 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: loading cluster ID for follower catalog manager: success
I20260808 00:42:44.204731 31388 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: acquiring CA information for follower catalog manager: success
I20260808 00:42:44.205122 31388 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 00:42:44.242507 31271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:44.242789 31271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:44.242818 31271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:44.244323 31271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:44.244382 31271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:44.244415 31271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:44.245957 30655 heartbeater.cc:503] Master 127.20.105.126:44365 was elected leader, sending a full tablet report...
I20260808 00:42:44.246542 31271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab
--principal=oryx/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:38339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=36153
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651,127.20.105.124:36103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:44.246935 31271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:44.247258 31271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:44.254921 31397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:44.255050 31395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:44.255209 31400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:44.255300 31271 server_base.cc:1034] running on GCE node
I20260808 00:42:44.255792 31271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:44.256405 31271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:44.257640 31271 hybrid_clock.cc:648] HybridClock initialized: now 1786149764257617 us; error 43 us; skew 500 ppm
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:44.260915 31271 init.cc:377] Logged in from keytab as oryx/127.20.105.65@KRBTEST.COM (short username oryx)
I20260808 00:42:44.261539 31271 webserver.cc:460] Webserver started at http://127.20.105.65:36153/ using document root <none> and password file <none>
I20260808 00:42:44.262044 31271 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:44.262177 31271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:44.263983 31271 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 00:42:44.264983 31407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.265226 31271 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:44.265350 31271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "a081ed75d561474bba9a154f25754026"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "43b1d0e56767e737e600a98230487323"
server_key_iv: "61954e3f6b225880a8c2ff9bbc9c8aad"
server_key_version: "encryptionkey@0"
I20260808 00:42:44.265861 31271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:44.276995 31271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:44.279822 31271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:44.280036 31271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:44.280614 31271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:44.281039 31271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:44.281098 31271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.281149 31271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:44.281181 31271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.293298 31271 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:38339
I20260808 00:42:44.293346 31566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:38339 every 8 connection(s)
I20260808 00:42:44.293998 31271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:44.300586 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 31271
I20260808 00:42:44.300822 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 30467
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.124@KRBTEST.COM
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.125@KRBTEST.COM
I20260808 00:42:44.310194 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.295250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:48315 (local address 127.20.105.125:43651)
0808 00:42:44.295336 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.295338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.295654 (+ 316us) server_negotiation.cc:407] Connection header received
0808 00:42:44.297372 (+ 1718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.297374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.297391 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.297452 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.298593 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.299486 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.300947 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.301306 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.301357 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.305818 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.305821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.305822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.305835 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.308679 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.309098 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.309099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.309099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.309158 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.309324 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.309325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.309325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.309498 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.309554 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.309941 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.310039 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:42:44.310662 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.295028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.20.105.65:48315)
0808 00:42:44.295563 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.295577 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.297213 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.297551 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.297556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.298141 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.298452 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.298456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.299617 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.299618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.300834 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.300837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.301470 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.301471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.301703 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.302742 (+ 1039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.302749 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.305661 (+ 2912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.308806 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.308807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.308808 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.309020 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.309215 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.309216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.309216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.309270 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.309584 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.309590 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.309795 (+ 205us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.310347 (+ 552us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.310475 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":70,"thread_start_us":118,"threads_started":1}
I20260808 00:42:44.310758 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.295028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:36103 (local address 127.20.105.65:45385)
0808 00:42:44.295313 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.295399 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.297210 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.297693 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.297694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.298173 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.298452 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.298456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.299835 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.299836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.300838 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.300841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.301498 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.301499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.301701 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.302583 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.302595 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.305654 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.308196 (+ 2542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.308207 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.308210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.308643 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.308930 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.308932 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.308932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.309176 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.309664 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.309666 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.309811 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.309928 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.310222 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":2,"thread_start_us":116,"threads_started":1}
I20260808 00:42:44.311233 31571 heartbeater.cc:347] Connected to a master server at 127.20.105.125:43651
I20260808 00:42:44.311334 31571 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.311359 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.295637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:45385 (local address 127.20.105.124:36103)
0808 00:42:44.295792 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.295795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.295805 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:42:44.297541 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.297542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.297583 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.297662 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.298604 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.299731 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.300986 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.301352 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.301423 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.305819 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.305821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.305822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.305837 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.308068 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.308743 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.308744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.308745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.308827 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.309284 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.309286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.309286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.309529 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.309624 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.310995 (+ 1371us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.311153 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260808 00:42:44.311559 31571 heartbeater.cc:511] Master 127.20.105.125:43651 requested a full tablet report, sending...
I20260808 00:42:44.312067 31567 heartbeater.cc:347] Connected to a master server at 127.20.105.124:36103
I20260808 00:42:44.312124 31567 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.312209 31567 heartbeater.cc:511] Master 127.20.105.124:36103 requested a full tablet report, sending...
I20260808 00:42:44.312839 31320 ts_manager.cc:194] Registered new tserver with Master: a081ed75d561474bba9a154f25754026 (127.20.105.65:38339)
I20260808 00:42:44.313436 31053 ts_manager.cc:194] Registered new tserver with Master: a081ed75d561474bba9a154f25754026 (127.20.105.65:38339)
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:44.333478 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.295539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:34717 (local address 127.20.105.126:44365)
0808 00:42:44.295606 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.295609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.296247 (+ 638us) server_negotiation.cc:407] Connection header received
0808 00:42:44.297402 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.297403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.297418 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.297452 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.298558 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.299234 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.326375 (+ 27141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.326738 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.326791 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.330158 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.330161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.330161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.330171 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.331879 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.332450 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.332451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.332451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.332512 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.332782 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.332783 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.332783 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.332944 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.333000 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.333235 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.333344 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:42:44.333496 31578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.295388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:34717)
0808 00:42:44.296142 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.296156 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.297234 (+ 1078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.297552 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.297556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.298026 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.298404 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.298410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.299339 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.299341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.326201 (+ 26860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.326212 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.326909 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.326910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.327014 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.327889 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.327895 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.330012 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.331982 (+ 1970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.331984 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.331985 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.332351 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.332602 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.332603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.332604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.332697 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.333019 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.333020 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.333090 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.333171 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.333320 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":8,"thread_start_us":106,"threads_started":1}
I20260808 00:42:44.333515 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:38391
--local_ip_for_outbound_sockets=127.20.105.66
--tserver_master_addrs=127.20.105.126:44365
--webserver_port=35243
--webserver_interface=127.20.105.66
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab
--principal=oryx/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651,127.20.105.124:36103 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:42:44.334036 31568 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44365
I20260808 00:42:44.334105 31568 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.334210 31568 heartbeater.cc:511] Master 127.20.105.126:44365 requested a full tablet report, sending...
I20260808 00:42:44.334609 30170 ts_manager.cc:194] Re-registered known tserver with Master: a081ed75d561474bba9a154f25754026 (127.20.105.65:38339)
I20260808 00:42:44.335172 30170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:34717
I20260808 00:42:44.427917 30848 heartbeater.cc:503] Master 127.20.105.126:44365 was elected leader, sending a full tablet report...
W20260808 00:42:44.435248 31582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:44.435554 31582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:44.435597 31582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:44.436954 31582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:44.437031 31582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:44.437078 31582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:44.438784 31582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab
--principal=oryx/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:38391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=35243
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651,127.20.105.124:36103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:44.439124 31582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:44.439419 31582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:44.445637 31590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:44.445647 31594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:44.445811 31591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:44.445968 31582 server_base.cc:1034] running on GCE node
I20260808 00:42:44.446216 31582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:44.446496 31582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:44.447661 31582 hybrid_clock.cc:648] HybridClock initialized: now 1786149764447637 us; error 46 us; skew 500 ppm
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:44.450932 31582 init.cc:377] Logged in from keytab as oryx/127.20.105.66@KRBTEST.COM (short username oryx)
I20260808 00:42:44.451478 31582 webserver.cc:460] Webserver started at http://127.20.105.66:35243/ using document root <none> and password file <none>
I20260808 00:42:44.451763 31582 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:44.451869 31582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:44.453720 31582 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:44.454629 31602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.454874 31582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:44.455003 31582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "5be4b9e066454bb3ba060bfb7145e179"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "d774928738c55d4a12b65d1820a6571c"
server_key_iv: "0418b46b3ec632273fbaf48d5c959e56"
server_key_version: "encryptionkey@0"
I20260808 00:42:44.455462 31582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:44.466430 31582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:44.468781 31582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:44.468984 31582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:44.469300 31582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:44.469710 31582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:44.469767 31582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.469817 31582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:44.469844 31582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.478121 31582 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:38391
I20260808 00:42:44.478188 31767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:38391 every 8 connection(s)
I20260808 00:42:44.478655 31582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:44.481248 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 31582
I20260808 00:42:44.481499 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 30660
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.125@KRBTEST.COM
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.124@KRBTEST.COM
I20260808 00:42:44.501638 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.479641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:37705 (local address 127.20.105.126:44365)
0808 00:42:44.479706 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.479710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.480066 (+ 356us) server_negotiation.cc:407] Connection header received
0808 00:42:44.481574 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.481575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.481590 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.481622 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.485363 (+ 3741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.485971 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.486868 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.487094 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.487130 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.496848 (+ 9718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.496851 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.496851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.496863 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.499206 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.500204 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.500205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.500206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.500276 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.500633 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.500634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.500634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.500818 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.500886 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.501387 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.501499 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:42:44.501704 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.479704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:47765 (local address 127.20.105.125:43651)
0808 00:42:44.479769 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.479771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.480296 (+ 525us) server_negotiation.cc:407] Connection header received
0808 00:42:44.481455 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.481458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.481477 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.481513 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.482916 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.484092 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.486367 (+ 2275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.486788 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.486845 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.497344 (+ 10499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.497346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.497347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.497359 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.499336 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.499937 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.499938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.499939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.500008 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.501063 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.501064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.501065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.501228 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.501289 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.501507 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.501602 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:42:44.502198 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.480157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:51371 (local address 127.20.105.124:36103)
0808 00:42:44.480235 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.480237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.480367 (+ 130us) server_negotiation.cc:407] Connection header received
0808 00:42:44.481445 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.481447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.481471 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.481510 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.489669 (+ 8159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.490535 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.491445 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.491682 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.491785 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.498294 (+ 6509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.498295 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.498296 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.498305 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.500762 (+ 2457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.501196 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.501197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.501198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.501265 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.501456 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.501457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.501457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.501622 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.501677 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.501921 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.502049 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:42:44.502468 31773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.479651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.20.105.66:47765)
0808 00:42:44.480193 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.480206 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.481312 (+ 1106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.481739 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.481743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.482318 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.482750 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.482759 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.484240 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.484242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.486236 (+ 1994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.486240 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.486998 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.486999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.487189 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.490169 (+ 2980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.490175 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.497218 (+ 7043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.499424 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.499425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.499426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.499806 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.500858 (+ 1052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.500859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.500860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.500971 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.501348 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.501349 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.501389 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.501854 (+ 465us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.502111 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
I20260808 00:42:44.502820 31772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.479651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:37705)
0808 00:42:44.479883 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.479928 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.481298 (+ 1370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.481738 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.481743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.485179 (+ 3436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.485265 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.485267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.486052 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.486053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.486790 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.486792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.487205 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.487206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.487260 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.489126 (+ 1866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.489139 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.496711 (+ 7572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.499302 (+ 2591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.499314 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.499317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.499775 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.500352 (+ 577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.500353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.500353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.500539 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.500912 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.500921 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.501204 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.502424 (+ 1220us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.502698 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":124,"threads_started":1}
I20260808 00:42:44.502936 31774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.480088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:36103 (local address 127.20.105.66:51371)
0808 00:42:44.480303 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.480311 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.481306 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.488919 (+ 7613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.488920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.489240 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.489512 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.489515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.490678 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.490679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.491342 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.491345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.491788 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.491789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.491944 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.492589 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.492594 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.497782 (+ 5188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.500841 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.500842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.500843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.501105 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.501333 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.501334 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.501334 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.501391 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.501717 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.501717 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.501770 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.502335 (+ 565us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.502767 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":196,"thread_start_us":75,"threads_started":1}
I20260808 00:42:44.503064 31770 heartbeater.cc:347] Connected to a master server at 127.20.105.125:43651
I20260808 00:42:44.503185 31770 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.503165 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:40329
--local_ip_for_outbound_sockets=127.20.105.67
--tserver_master_addrs=127.20.105.126:44365
--webserver_port=40347
--webserver_interface=127.20.105.67
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab
--principal=oryx/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:34327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651,127.20.105.124:36103 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:42:44.503440 31770 heartbeater.cc:511] Master 127.20.105.125:43651 requested a full tablet report, sending...
I20260808 00:42:44.503639 31768 heartbeater.cc:347] Connected to a master server at 127.20.105.124:36103
I20260808 00:42:44.503692 31768 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.503784 31768 heartbeater.cc:511] Master 127.20.105.124:36103 requested a full tablet report, sending...
I20260808 00:42:44.503696 31769 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44365
I20260808 00:42:44.503906 31769 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.503966 31769 heartbeater.cc:511] Master 127.20.105.126:44365 requested a full tablet report, sending...
I20260808 00:42:44.504076 31053 ts_manager.cc:194] Registered new tserver with Master: 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391)
I20260808 00:42:44.504146 31320 ts_manager.cc:194] Registered new tserver with Master: 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391)
I20260808 00:42:44.504226 30170 ts_manager.cc:194] Re-registered known tserver with Master: 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391)
I20260808 00:42:44.504698 30170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:37705
W20260808 00:42:44.601958 31775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:44.602233 31775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:44.602260 31775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:44.603582 31775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:44.603633 31775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:44.603662 31775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:44.605243 31775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab
--principal=oryx/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:40329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=40347
--tserver_master_addrs=127.20.105.126:44365,127.20.105.125:43651,127.20.105.124:36103
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:44.605492 31775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:44.605716 31775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:44.611974 31780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:44.611979 31784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:44.611969 31781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:44.612483 31775 server_base.cc:1034] running on GCE node
I20260808 00:42:44.612725 31775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:44.613005 31775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:44.614207 31775 hybrid_clock.cc:648] HybridClock initialized: now 1786149764614194 us; error 36 us; skew 500 ppm
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:44.617290 31775 init.cc:377] Logged in from keytab as oryx/127.20.105.67@KRBTEST.COM (short username oryx)
I20260808 00:42:44.617926 31775 webserver.cc:460] Webserver started at http://127.20.105.67:40347/ using document root <none> and password file <none>
I20260808 00:42:44.618193 31775 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:44.618288 31775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:44.619930 31775 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:44.620698 31796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.620903 31775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:44.621011 31775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "43c44f6b5ab243b59df0aaa532bb86ff"
format_stamp: "Formatted at 2026-08-08 00:42:43 on dist-test-slave-kvfs"
server_key: "919a7743f6c03299387052ff68344f04"
server_key_iv: "5c3a455fc8847063713b8308451c7aa6"
server_key_version: "encryptionkey@0"
I20260808 00:42:44.621608 31775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:44.624341 31797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.620652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:36103 (local address 127.0.0.1:50958)
0808 00:42:44.620815 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.620828 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.620965 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.621204 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.621205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.621208 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:44.621281 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.621283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.622270 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.622271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.623089 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.623091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.623998 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.623999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.624140 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.624145 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.624206 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.624263 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":69,"threads_started":1}
I20260808 00:42:44.624369 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.620751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.20.105.124:36103)
0808 00:42:44.620822 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.620825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.620900 (+ 75us) server_negotiation.cc:407] Connection header received
0808 00:42:44.621074 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.621075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.621111 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.621193 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:44.621350 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.622181 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.623184 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.623863 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.623961 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.624089 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.624228 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.624285 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:42:44.624747 31336 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 00:42:44.624974 31372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 837aae531f9e489d97bb37425d5df64e. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.625108 31372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:42:44.636907 31775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:44.639704 31775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:44.639884 31775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:44.640260 31775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:44.640689 31775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:44.640746 31775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.640791 31775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:44.640820 31775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:44.649372 31775 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:40329
I20260808 00:42:44.649432 31953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:40329 every 8 connection(s)
I20260808 00:42:44.649930 31775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:44.651073 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 31775
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.125@KRBTEST.COM
Aug 08 00:42:44 dist-test-slave-kvfs krb5kdc[30102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149764, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.124@KRBTEST.COM
I20260808 00:42:44.666172 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.652948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47338 (local address 127.20.105.125:43651)
0808 00:42:44.653431 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.653434 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.653854 (+ 420us) server_negotiation.cc:407] Connection header received
0808 00:42:44.653885 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.653886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.653903 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.654964 (+ 1061us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.655017 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.655872 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.660369 (+ 4497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.660741 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.661145 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.662398 (+ 1253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.662400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.662400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.662408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.664392 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.664843 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.664844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.664845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.664907 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.665163 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.665164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.665164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.665399 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.665867 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.665932 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.666052 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":137,"threads_started":1}
I20260808 00:42:44.666532 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.651136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:33843 (local address 127.20.105.125:43651)
0808 00:42:44.651288 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.651290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.651956 (+ 666us) server_negotiation.cc:407] Connection header received
0808 00:42:44.652728 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.652731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.652750 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.652806 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.654056 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.654894 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.656697 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.657012 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.657057 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.661412 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.661414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.661415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.661424 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.663467 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.665531 (+ 2064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.665532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.665532 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.665592 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.665821 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.665822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.665822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.665952 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.666027 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.666357 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.666435 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260808 00:42:44.666828 31958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.651076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.20.105.67:33843)
0808 00:42:44.651849 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.651859 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.652584 (+ 725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.653006 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.653012 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.653669 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.653956 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.653958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.655041 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.655042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.656591 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.656595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.657145 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.657146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.657325 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.658131 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.659062 (+ 931us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.661295 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.663577 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.663589 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.663593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.663976 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.665668 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.665669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.665669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.665742 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.666057 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.666058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.666207 (+ 149us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.666326 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.666535 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"mutex_wait_us":550,"thread_start_us":86,"threads_started":1}
I20260808 00:42:44.666841 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.651430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:39641)
0808 00:42:44.651834 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.651856 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.652584 (+ 728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.653006 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.653012 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.653586 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.653939 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.653946 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.655046 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.655047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.656591 (+ 1544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.656595 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.657137 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.657139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.657325 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.658180 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.658185 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.660388 (+ 2203us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.664775 (+ 4387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.664775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.664776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.665067 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.665337 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.665338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.665338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.665504 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.665979 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.665988 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.666208 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.666594 (+ 386us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.666717 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":95,"thread_start_us":97,"threads_started":1}
I20260808 00:42:44.667660 31956 heartbeater.cc:347] Connected to a master server at 127.20.105.125:43651
I20260808 00:42:44.667794 31956 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.668093 31956 heartbeater.cc:511] Master 127.20.105.125:43651 requested a full tablet report, sending...
I20260808 00:42:44.668403 30233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.651579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:39641 (local address 127.20.105.126:44365)
0808 00:42:44.651642 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.651645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.651939 (+ 294us) server_negotiation.cc:407] Connection header received
0808 00:42:44.652778 (+ 839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.652780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.652795 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.652846 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.654087 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.654893 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.656696 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.657009 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.657068 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.660502 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.660504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.660504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.660513 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.664639 (+ 4126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.665189 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.665191 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.665191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.665265 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.665665 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.665666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.665667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.665849 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.665919 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.668175 (+ 2256us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.668284 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:42:44.668781 31053 ts_manager.cc:194] Registered new tserver with Master: 43c44f6b5ab243b59df0aaa532bb86ff (127.20.105.67:40329)
I20260808 00:42:44.669109 31955 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44365
I20260808 00:42:44.669157 31955 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.669232 31955 heartbeater.cc:511] Master 127.20.105.126:44365 requested a full tablet report, sending...
I20260808 00:42:44.669525 30170 ts_manager.cc:194] Re-registered known tserver with Master: 43c44f6b5ab243b59df0aaa532bb86ff (127.20.105.67:40329)
I20260808 00:42:44.669704 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.653333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50966 (local address 127.20.105.124:36103)
0808 00:42:44.653683 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.653685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.654000 (+ 315us) server_negotiation.cc:407] Connection header received
0808 00:42:44.654128 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.654129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.654146 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.654197 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.655122 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.655988 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.657270 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.658237 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.658288 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.662245 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.662247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.662247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.662256 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.664621 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.668606 (+ 3985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.668608 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.668608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.668681 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.668903 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.668904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.668904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.669055 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.669137 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.669370 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.669461 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":95,"threads_started":1}
I20260808 00:42:44.670090 30170 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:39641
I20260808 00:42:44.671046 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.652771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49006 (local address 127.20.105.126:44365)
0808 00:42:44.653360 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.653364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.653662 (+ 298us) server_negotiation.cc:407] Connection header received
0808 00:42:44.653695 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.653696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.653715 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.653804 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.656775 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.657613 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.659821 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.660202 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.660279 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.665684 (+ 5405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.665686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.665686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.665693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.669538 (+ 3845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.669911 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.669913 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.669913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.669982 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.670223 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.670224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.670224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.670367 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.670474 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.670861 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.670956 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":84,"threads_started":1}
I20260808 00:42:44.671823 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.651217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:51369 (local address 127.20.105.124:36103)
0808 00:42:44.651292 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.651295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.651906 (+ 611us) server_negotiation.cc:407] Connection header received
0808 00:42:44.657664 (+ 5758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.657666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.657685 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.657766 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.658132 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.658830 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.660090 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.660380 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.660442 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.667448 (+ 7006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.667450 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.667450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.667458 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.669314 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.670741 (+ 1427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.670742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.670743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.670800 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.671033 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.671034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.671034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.671188 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.671250 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.671656 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.671752 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:42:44.671861 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.651076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:36103 (local address 127.20.105.67:51369)
0808 00:42:44.651726 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.651791 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.655335 (+ 3544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.657771 (+ 2436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.657772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.657949 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:44.658036 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.658038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.658939 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.658940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.659929 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.659932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.660527 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.660528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.660599 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.664121 (+ 3522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:44.664127 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:44.667334 (+ 3207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:44.670487 (+ 3153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.670487 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.670488 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.670651 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.670878 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:44.670878 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:44.670879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:44.670951 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:44.671474 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:44.671475 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:44.671523 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.671589 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.671710 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":95,"threads_started":1}
I20260808 00:42:44.672184 31954 heartbeater.cc:347] Connected to a master server at 127.20.105.124:36103
I20260808 00:42:44.672238 31954 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:44.672322 31954 heartbeater.cc:511] Master 127.20.105.124:36103 requested a full tablet report, sending...
I20260808 00:42:44.672644 31320 ts_manager.cc:194] Registered new tserver with Master: 43c44f6b5ab243b59df0aaa532bb86ff (127.20.105.67:40329)
I20260808 00:42:44.673046 30170 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49006:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:44.675187 30170 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:44.676391 31795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: NON_VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: true } } }
I20260808 00:42:44.676505 31795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:44.679083 31795 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: attempting to promote NON_VOTER 700e8c77909849b49ea4c79ccd7ac037 to VOTER
I20260808 00:42:44.679235 31795 consensus_queue.cc:237] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.9, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } }
I20260808 00:42:44.679690 31336 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 LEARNER]: Refusing update from remote peer 837aae531f9e489d97bb37425d5df64e: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260808 00:42:44.679826 31064 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Refusing update from remote peer 837aae531f9e489d97bb37425d5df64e: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260808 00:42:44.680140 31977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Connected to new peer: Peer: permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 00:42:44.680308 31979 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [LEADER]: Connected to new peer: Peer: permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 00:42:44.680907 31977 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [term 1 LEADER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:44.681061 31064 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:44.682451 31977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:44.682554 31977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:44.682618 31977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:44.682688 31977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:44.683631 31795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:44.683632 31996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:44.683725 31795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:44.683727 31996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:42:44.683972 31336 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Committing config change with OpId 1.10: config changed from index 7 to 10, 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124) changed from NON_VOTER to VOTER. New config: { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:44.685638 31372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "837aae531f9e489d97bb37425d5df64e" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:44.685748 31372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:42:44.686914 31993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.681886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:38391 (local address 127.0.0.1:56218)
0808 00:42:44.682086 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.682097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.682150 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.683428 (+ 1278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.683430 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.683432 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:44.683490 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.683492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.684740 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.684742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.685882 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.685885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.686647 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.686648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.686711 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.686716 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.686773 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.686826 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":72,"threads_started":1}
I20260808 00:42:44.686934 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.682801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.20.105.66:38391)
0808 00:42:44.683224 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.683229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.683241 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:44.683292 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.683295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.683313 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.683570 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:44.683645 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.684648 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.685964 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.686548 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.686648 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.686740 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.686801 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.686844 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":139,"threads_started":1}
I20260808 00:42:44.687598 31797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.681609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:40329 (local address 127.0.0.1:49340)
0808 00:42:44.681656 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.681672 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.681722 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.682836 (+ 1114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.682837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.682840 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:44.682915 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.682918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.684629 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.684630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.685568 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.685571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.687308 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.687309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.687375 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.687379 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.687448 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.687501 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":21}
I20260808 00:42:44.687619 31995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.682229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.20.105.67:40329)
0808 00:42:44.682492 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.682497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.682510 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:42:44.682697 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.682700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.682725 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.682839 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:44.683120 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.684520 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.686030 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.686892 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.687269 (+ 377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.687365 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.687463 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.687512 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":104,"threads_started":1}
I20260808 00:42:44.687973 31670 tablet_service.cc:1467] Processing CreateTablet for tablet 6e70d4dfdee4442a8703230a4a0af6fe (DEFAULT_TABLE table=test-table [id=b76f2d9ffb8142b48f09b43044aa8d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:44.688577 31670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e70d4dfdee4442a8703230a4a0af6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:44.688524 31865 tablet_service.cc:1467] Processing CreateTablet for tablet 6e70d4dfdee4442a8703230a4a0af6fe (DEFAULT_TABLE table=test-table [id=b76f2d9ffb8142b48f09b43044aa8d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:44.688882 31865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e70d4dfdee4442a8703230a4a0af6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:44.690292 31991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.681706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:38339 (local address 127.0.0.1:48544)
0808 00:42:44.681846 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.681855 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.681896 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.686008 (+ 4112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.686009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.686012 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:44.686072 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.686074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.687914 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.687915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.689070 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.689074 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.690004 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.690005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.690074 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.690079 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.690111 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.690158 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":102,"threads_started":1}
I20260808 00:42:44.690817 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.682483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.20.105.65:38339)
0808 00:42:44.685794 (+ 3311us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.685799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.685810 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:44.685860 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.685863 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.685880 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.686200 (+ 320us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:44.686283 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.687802 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.689183 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.689903 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.690488 (+ 585us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.690624 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.690654 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.690703 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1337,"thread_start_us":109,"threads_started":1}
I20260808 00:42:44.691756 31477 tablet_service.cc:1467] Processing CreateTablet for tablet 6e70d4dfdee4442a8703230a4a0af6fe (DEFAULT_TABLE table=test-table [id=b76f2d9ffb8142b48f09b43044aa8d19]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:44.692281 31477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6e70d4dfdee4442a8703230a4a0af6fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:44.693077 32002 tablet_bootstrap.cc:492] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Bootstrap starting.
I20260808 00:42:44.693442 32005 tablet_bootstrap.cc:492] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: Bootstrap starting.
I20260808 00:42:44.694041 32002 tablet_bootstrap.cc:654] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:44.694213 32005 tablet_bootstrap.cc:654] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:44.694598 32005 log.cc:824] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:44.695171 32002 log.cc:824] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:44.695629 32005 tablet_bootstrap.cc:492] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: No bootstrap required, opened a new log
I20260808 00:42:44.695708 32005 ts_tablet_manager.cc:1397] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:42:44.696300 32002 tablet_bootstrap.cc:492] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: No bootstrap required, opened a new log
I20260808 00:42:44.696396 32002 ts_tablet_manager.cc:1397] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:42:44.697009 32009 tablet_bootstrap.cc:492] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Bootstrap starting.
I20260808 00:42:44.698477 32009 tablet_bootstrap.cc:654] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:44.699038 32009 log.cc:824] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:44.699200 32005 raft_consensus.cc:348] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.699425 32005 raft_consensus.cc:374] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:44.699476 32005 raft_consensus.cc:729] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c44f6b5ab243b59df0aaa532bb86ff, State: Initialized, Role: FOLLOWER
I20260808 00:42:44.699664 32005 consensus_queue.cc:260] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.700038 31955 heartbeater.cc:503] Master 127.20.105.126:44365 was elected leader, sending a full tablet report...
I20260808 00:42:44.699993 32005 ts_tablet_manager.cc:1428] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 00:42:44.701264 32002 raft_consensus.cc:348] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.701490 32002 raft_consensus.cc:374] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:44.701633 32002 raft_consensus.cc:729] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be4b9e066454bb3ba060bfb7145e179, State: Initialized, Role: FOLLOWER
I20260808 00:42:44.701807 32002 consensus_queue.cc:260] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.702189 31769 heartbeater.cc:503] Master 127.20.105.126:44365 was elected leader, sending a full tablet report...
I20260808 00:42:44.702170 32002 ts_tablet_manager.cc:1428] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260808 00:42:44.702378 32009 tablet_bootstrap.cc:492] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: No bootstrap required, opened a new log
I20260808 00:42:44.702473 32009 ts_tablet_manager.cc:1397] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 00:42:44.705713 32009 raft_consensus.cc:348] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.705865 32009 raft_consensus.cc:374] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:44.705890 32009 raft_consensus.cc:729] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a081ed75d561474bba9a154f25754026, State: Initialized, Role: FOLLOWER
I20260808 00:42:44.705981 32009 consensus_queue.cc:260] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.706213 32009 ts_tablet_manager.cc:1428] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:42:44.706310 31568 heartbeater.cc:503] Master 127.20.105.126:44365 was elected leader, sending a full tablet report...
W20260808 00:42:44.730007 31771 tablet.cc:2369] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:42:44.795630 31575 tablet.cc:2369] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:44.830706 32012 raft_consensus.cc:482] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:44.830909 32012 raft_consensus.cc:504] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.831508 32012 leader_election.cc:288] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a081ed75d561474bba9a154f25754026 (127.20.105.65:38339), 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391)
I20260808 00:42:44.835564 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.831771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:38339 (local address 127.20.105.67:43605)
0808 00:42:44.831868 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.831886 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.831959 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.832172 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.832174 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.832177 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:44.832284 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.832287 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.833061 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.833063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.834328 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.834331 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.835248 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.835249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.835331 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.835337 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.835398 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.835452 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 00:42:44.835573 31999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.831883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:43605 (local address 127.20.105.65:38339)
0808 00:42:44.831980 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.831983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.831994 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:44.832067 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.832068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.832086 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.832157 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:44.832378 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.832965 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.834435 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.835112 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.835212 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.835225 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.835424 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.835473 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:42:44.836135 31502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70d4dfdee4442a8703230a4a0af6fe" candidate_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a081ed75d561474bba9a154f25754026" is_pre_election: true
I20260808 00:42:44.836381 31502 raft_consensus.cc:2454] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43c44f6b5ab243b59df0aaa532bb86ff in term 0.
I20260808 00:42:44.836567 31958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.831988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:38391 (local address 127.20.105.67:47619)
0808 00:42:44.832073 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:44.832083 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:44.832115 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:44.832288 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:44.832289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:44.832291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:44.832410 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.832412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.834060 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.834061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.835468 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:44.835472 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.836286 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.836287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:44.836368 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.836371 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:44.836440 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:42:44.836489 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 00:42:44.836568 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.832062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:47619 (local address 127.20.105.66:38391)
0808 00:42:44.832141 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.832144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.832154 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:42:44.832186 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.832187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.832197 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.832270 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:44.832501 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.833951 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.835582 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.836183 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.836275 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.836286 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.836448 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.836494 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:42:44.836683 31798 leader_election.cc:302] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43c44f6b5ab243b59df0aaa532bb86ff, a081ed75d561474bba9a154f25754026; no voters:
I20260808 00:42:44.836916 32012 raft_consensus.cc:2781] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:44.837010 32012 raft_consensus.cc:482] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:44.837041 32012 raft_consensus.cc:3037] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:44.837179 31703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70d4dfdee4442a8703230a4a0af6fe" candidate_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be4b9e066454bb3ba060bfb7145e179" is_pre_election: true
I20260808 00:42:44.837376 31703 raft_consensus.cc:2454] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43c44f6b5ab243b59df0aaa532bb86ff in term 0.
I20260808 00:42:44.838429 32012 raft_consensus.cc:504] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.838569 32012 leader_election.cc:288] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [CANDIDATE]: Term 1 election: Requested vote from peers a081ed75d561474bba9a154f25754026 (127.20.105.65:38339), 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391)
I20260808 00:42:44.838816 31502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70d4dfdee4442a8703230a4a0af6fe" candidate_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a081ed75d561474bba9a154f25754026"
I20260808 00:42:44.838857 31703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6e70d4dfdee4442a8703230a4a0af6fe" candidate_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be4b9e066454bb3ba060bfb7145e179"
I20260808 00:42:44.838927 31502 raft_consensus.cc:3037] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:44.839011 31703 raft_consensus.cc:3037] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:44.839833 31502 raft_consensus.cc:2454] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43c44f6b5ab243b59df0aaa532bb86ff in term 1.
I20260808 00:42:44.840059 31798 leader_election.cc:302] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43c44f6b5ab243b59df0aaa532bb86ff, a081ed75d561474bba9a154f25754026; no voters:
I20260808 00:42:44.840154 31703 raft_consensus.cc:2454] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43c44f6b5ab243b59df0aaa532bb86ff in term 1.
I20260808 00:42:44.840222 32012 raft_consensus.cc:2781] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:44.840466 32012 raft_consensus.cc:686] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 LEADER]: Becoming Leader. State: Replica: 43c44f6b5ab243b59df0aaa532bb86ff, State: Running, Role: LEADER
I20260808 00:42:44.840600 32012 consensus_queue.cc:237] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:44.841776 30170 catalog_manager.cc:5495] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 43c44f6b5ab243b59df0aaa532bb86ff (127.20.105.67). New cstate: current_term: 1 leader_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:44.872593 31995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.869713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49346 (local address 127.20.105.67:40329)
0808 00:42:44.869781 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.869784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.869801 (+ 17us) server_negotiation.cc:407] Connection header received
0808 00:42:44.869854 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.869855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.869880 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.869966 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:44.870164 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.870765 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.871668 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.871879 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.871948 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.872189 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:44.872337 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:44.872417 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.872480 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.872509 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:42:44.875639 31502 raft_consensus.cc:1261] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 1 FOLLOWER]: Refusing update from remote peer 43c44f6b5ab243b59df0aaa532bb86ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:44.875803 31703 raft_consensus.cc:1261] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 1 FOLLOWER]: Refusing update from remote peer 43c44f6b5ab243b59df0aaa532bb86ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:44.876214 32012 consensus_queue.cc:1035] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:44.876464 32015 consensus_queue.cc:1035] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:44.882911 32020 mvcc.cc:205] Tried to move back new op lower bound from 7316069436926554112 to 7316069436787945472. Current Snapshot: MvccSnapshot[applied={T|T < 7316069436926554112}]
I20260808 00:42:44.885046 30167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:44.885196 30167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49006:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:44.887265 30167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 837aae531f9e489d97bb37425d5df64e: Sending DeleteTablet for 3 replicas of tablet 6e70d4dfdee4442a8703230a4a0af6fe
I20260808 00:42:44.887814 31865 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e70d4dfdee4442a8703230a4a0af6fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:44 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:49340
I20260808 00:42:44.887857 31477 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e70d4dfdee4442a8703230a4a0af6fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:44 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:48544
I20260808 00:42:44.888136 31670 tablet_service.cc:1514] Processing DeleteTablet for tablet 6e70d4dfdee4442a8703230a4a0af6fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:44 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:56218
I20260808 00:42:44.888548 32035 tablet_replica.cc:330] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: stopping tablet replica
I20260808 00:42:44.888697 32034 tablet_replica.cc:330] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: stopping tablet replica
I20260808 00:42:44.888723 32035 raft_consensus.cc:2229] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:44.888890 32034 raft_consensus.cc:2229] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:44.888928 32035 raft_consensus.cc:2258] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:44.889279 32034 raft_consensus.cc:2258] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:44.889492 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 30123
I20260808 00:42:44.889580 32035 ts_tablet_manager.cc:1905] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:44.899282 32034 ts_tablet_manager.cc:1905] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:44.901598 32035 ts_tablet_manager.cc:1918] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:44.901736 32035 log.cc:1196] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/6e70d4dfdee4442a8703230a4a0af6fe
I20260808 00:42:44.902185 32035 ts_tablet_manager.cc:1939] T 6e70d4dfdee4442a8703230a4a0af6fe P a081ed75d561474bba9a154f25754026: Deleting consensus metadata
I20260808 00:42:44.902449 32034 ts_tablet_manager.cc:1918] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:44.902619 32034 log.cc:1196] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/6e70d4dfdee4442a8703230a4a0af6fe
I20260808 00:42:44.902973 32034 ts_tablet_manager.cc:1939] T 6e70d4dfdee4442a8703230a4a0af6fe P 43c44f6b5ab243b59df0aaa532bb86ff: Deleting consensus metadata
I20260808 00:42:44.904677 32036 tablet_replica.cc:330] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: stopping tablet replica
I20260808 00:42:44.904939 32036 raft_consensus.cc:2229] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:44.905211 32036 raft_consensus.cc:2258] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:44.906376 32036 ts_tablet_manager.cc:1905] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:44.908939 32036 ts_tablet_manager.cc:1918] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:44.909032 32036 log.cc:1196] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/6e70d4dfdee4442a8703230a4a0af6fe
I20260808 00:42:44.909332 32036 ts_tablet_manager.cc:1939] T 6e70d4dfdee4442a8703230a4a0af6fe P 5be4b9e066454bb3ba060bfb7145e179: Deleting consensus metadata
W20260808 00:42:44.910954 31411 connection.cc:535] server connection from 127.0.0.1:48544 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 00:42:44.911291 31801 connection.cc:535] server connection from 127.0.0.1:49340 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 00:42:44.912086 31609 connection.cc:535] server connection from 127.0.0.1:56218 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:42:44.915568 31580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.888448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50972 (local address 127.20.105.124:36103)
0808 00:42:44.888500 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.888503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.904446 (+ 15943us) server_negotiation.cc:407] Connection header received
0808 00:42:44.904542 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.904544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.904564 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.904645 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.905703 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.906654 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.908084 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.908416 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.908471 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.911559 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.911563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.911564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.911575 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.913862 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.914345 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.914347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.914348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.914413 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.914797 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.914798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.914799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.914993 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.915093 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.915338 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.915436 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:42:44.923530 31129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.914710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47342 (local address 127.20.105.125:43651)
0808 00:42:44.914769 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.914772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.914780 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:44.914811 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.914811 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.914825 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.914874 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.915913 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.916768 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.917967 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.918180 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.918267 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.920415 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.920416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.920417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.920434 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.922171 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.922569 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.922570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.922571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.922638 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.922851 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.922852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.922852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.923047 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.923158 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.923295 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.923407 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:42:44.923852 31973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:44.914691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.20.105.124:36103)
0808 00:42:44.914753 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:44.914755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:44.914763 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:44.914789 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:44.914790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:44.914806 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:44.914861 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:44.915918 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.916767 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.917953 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:44.918337 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:44.918422 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:44.920382 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:44.920385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:44.920385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:44.920395 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:44.922501 (+ 2106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.922942 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.922943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.922943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.923002 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:44.923231 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:44.923232 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:44.923232 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:44.923382 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:44.923486 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:44.923643 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 00:42:44.923747 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:42:45.883116 32044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:45.882600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:38663)
0808 00:42:45.882781 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:45.883046 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":104,"threads_started":1}
I20260808 00:42:45.883116 32045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:45.882730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:40213)
0808 00:42:45.882903 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:45.883028 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":141,"threads_started":1}
W20260808 00:42:45.883527 31568 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:44365 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
W20260808 00:42:45.883528 31955 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:44365 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:45.886040 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:45.885750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:60681)
0808 00:42:45.885903 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:45.885990 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":71,"threads_started":1}
W20260808 00:42:45.886341 31769 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:44365 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:46.387733 32049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 837aae531f9e489d97bb37425d5df64e)
I20260808 00:42:46.388127 32050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 837aae531f9e489d97bb37425d5df64e)
I20260808 00:42:46.388347 32050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } }
I20260808 00:42:46.388345 32049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } }
I20260808 00:42:46.388972 32049 leader_election.cc:288] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365), 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124:36103)
I20260808 00:42:46.389503 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.389161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49512)
0808 00:42:46.389394 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.389453 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":112,"threads_started":1}
I20260808 00:42:46.389493 32050 leader_election.cc:288] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365), 707d92be9cbc45278e303eff39258662 (127.20.105.125:43651)
I20260808 00:42:46.389505 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.389176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49528)
0808 00:42:46.389373 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.389452 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":109,"threads_started":1}
W20260808 00:42:46.389827 31032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
W20260808 00:42:46.390206 31302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:46.390679 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.390534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49544)
0808 00:42:46.390592 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.390633 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:42:46.390779 31032 leader_election.cc:334] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365): Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:46.390923 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.390650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49550)
0808 00:42:46.390694 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.390731 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 00:42:46.391013 31302 leader_election.cc:334] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365): Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:46.393811 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.389396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:36103 (local address 127.0.0.1:50988)
0808 00:42:46.389611 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.389624 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:46.389702 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:46.390043 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:46.390045 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:46.390050 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:46.390138 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:46.390142 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:46.391184 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:46.391185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:46.392563 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:46.392567 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:46.393449 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:46.393450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:46.393530 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:46.393536 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:42:46.393614 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 00:42:46.393671 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":56,"threads_started":1}
I20260808 00:42:46.393814 32055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.389390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47348 (local address 127.20.105.125:43651)
0808 00:42:46.389696 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:46.389700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:46.389963 (+ 263us) server_negotiation.cc:407] Connection header received
0808 00:42:46.390002 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:46.390002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:46.390014 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:46.390106 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:46.390299 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:46.391164 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:46.392692 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:46.393323 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:46.393419 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:46.393432 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:46.393640 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 00:42:46.393693 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":69,"threads_started":1}
I20260808 00:42:46.393800 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.389639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.20.105.124:36103)
0808 00:42:46.389782 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:46.389784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:46.389796 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:46.389837 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:46.389838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:46.389854 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:46.389961 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:46.390228 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:46.391091 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:46.392692 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:46.393323 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:46.393420 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:46.393432 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:46.393640 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 00:42:46.393693 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":61,"threads_started":1}
I20260808 00:42:46.393811 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.389221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:43651 (local address 127.0.0.1:47348)
0808 00:42:46.389826 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.389838 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:46.389932 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:46.390113 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:46.390114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:46.390119 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:46.390199 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:46.390202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:46.391241 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:46.391242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:46.392563 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:46.392566 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:46.393450 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:46.393450 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:46.393530 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:46.393536 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:42:46.393613 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 00:42:46.393671 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":73,"threads_started":1}
I20260808 00:42:46.394284 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "707d92be9cbc45278e303eff39258662" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "700e8c77909849b49ea4c79ccd7ac037" is_pre_election: true
I20260808 00:42:46.394480 31336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 707d92be9cbc45278e303eff39258662 in term 1.
I20260808 00:42:46.394490 31064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "700e8c77909849b49ea4c79ccd7ac037" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "707d92be9cbc45278e303eff39258662" is_pre_election: true
I20260808 00:42:46.394627 31064 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 700e8c77909849b49ea4c79ccd7ac037 in term 1.
I20260808 00:42:46.394657 31032 leader_election.cc:302] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 700e8c77909849b49ea4c79ccd7ac037, 707d92be9cbc45278e303eff39258662; no voters: 837aae531f9e489d97bb37425d5df64e
I20260808 00:42:46.394763 32049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 00:42:46.394810 32049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 837aae531f9e489d97bb37425d5df64e)
I20260808 00:42:46.394840 32049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:42:46.395828 32049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } }
I20260808 00:42:46.396004 32049 leader_election.cc:288] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 2 election: Requested vote from peers 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365), 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124:36103)
I20260808 00:42:46.396174 31300 leader_election.cc:302] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 700e8c77909849b49ea4c79ccd7ac037, 707d92be9cbc45278e303eff39258662; no voters: 837aae531f9e489d97bb37425d5df64e
I20260808 00:42:46.396308 32050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 00:42:46.396370 32050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 837aae531f9e489d97bb37425d5df64e)
I20260808 00:42:46.396386 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.396264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49554)
0808 00:42:46.396309 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.396352 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 00:42:46.396409 32050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:42:46.396438 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "707d92be9cbc45278e303eff39258662" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "700e8c77909849b49ea4c79ccd7ac037"
I20260808 00:42:46.396992 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.396887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49570)
0808 00:42:46.396937 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.396968 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:42:46.397065 31032 leader_election.cc:334] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365): Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:46.397634 32050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } }
I20260808 00:42:46.397811 32050 leader_election.cc:288] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [CANDIDATE]: Term 2 election: Requested vote from peers 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365), 707d92be9cbc45278e303eff39258662 (127.20.105.125:43651)
I20260808 00:42:46.397820 31336 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 707d92be9cbc45278e303eff39258662 in current term 2: Already voted for candidate 700e8c77909849b49ea4c79ccd7ac037 in this term.
I20260808 00:42:46.397986 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.397887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49576)
0808 00:42:46.397931 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.397958 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:42:46.398088 31064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "700e8c77909849b49ea4c79ccd7ac037" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "707d92be9cbc45278e303eff39258662"
I20260808 00:42:46.398201 31064 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 700e8c77909849b49ea4c79ccd7ac037 in current term 2: Already voted for candidate 707d92be9cbc45278e303eff39258662 in this term.
I20260808 00:42:46.398329 32052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.398224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49590)
0808 00:42:46.398269 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.398297 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 00:42:46.398502 31302 leader_election.cc:334] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365): Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:46.398555 31302 leader_election.cc:302] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 700e8c77909849b49ea4c79ccd7ac037; no voters: 707d92be9cbc45278e303eff39258662, 837aae531f9e489d97bb37425d5df64e
I20260808 00:42:46.398542 31032 leader_election.cc:302] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 707d92be9cbc45278e303eff39258662; no voters: 700e8c77909849b49ea4c79ccd7ac037, 837aae531f9e489d97bb37425d5df64e
I20260808 00:42:46.398658 32050 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 00:42:46.398772 32049 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 00:42:46.884527 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.884167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:60483)
0808 00:42:46.884371 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.884433 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":124,"threads_started":1}
I20260808 00:42:46.884562 32062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.884172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:50155)
0808 00:42:46.884440 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.884489 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":97,"threads_started":1}
I20260808 00:42:46.887036 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:46.886731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:52895)
0808 00:42:46.886915 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:46.886956 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":80,"threads_started":1}
I20260808 00:42:47.885650 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:47.885289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:36141)
0808 00:42:47.885483 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:47.885559 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
I20260808 00:42:47.885668 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:47.885299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:45579)
0808 00:42:47.885566 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:47.885610 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":102,"threads_started":1}
I20260808 00:42:47.888199 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:47.887654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:49997)
0808 00:42:47.887846 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:47.887929 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":130,"threads_started":1}
I20260808 00:42:48.299209 32074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:48.299367 32074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } }
I20260808 00:42:48.299664 32074 leader_election.cc:288] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365), 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124:36103)
I20260808 00:42:48.300204 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.299909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49808)
0808 00:42:48.300087 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.300144 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
I20260808 00:42:48.300354 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "707d92be9cbc45278e303eff39258662" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "700e8c77909849b49ea4c79ccd7ac037" is_pre_election: true
I20260808 00:42:48.300616 31336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 707d92be9cbc45278e303eff39258662 in term 2.
I20260808 00:42:48.300864 31032 leader_election.cc:302] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 700e8c77909849b49ea4c79ccd7ac037, 707d92be9cbc45278e303eff39258662; no voters:
I20260808 00:42:48.300925 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.300785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49812)
0808 00:42:48.300839 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.300874 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:42:48.300974 31032 leader_election.cc:334] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365): Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:48.301054 32074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 00:42:48.301119 32074 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:48.301141 32074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:42:48.302435 32074 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } }
I20260808 00:42:48.302614 32074 leader_election.cc:288] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 3 election: Requested vote from peers 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365), 700e8c77909849b49ea4c79ccd7ac037 (127.20.105.124:36103)
I20260808 00:42:48.302870 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.302747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49816)
0808 00:42:48.302811 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.302846 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:42:48.302947 31336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "707d92be9cbc45278e303eff39258662" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "700e8c77909849b49ea4c79ccd7ac037"
I20260808 00:42:48.303067 31336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:42:48.303282 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.303187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49818)
0808 00:42:48.303237 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.303264 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 00:42:48.303367 31032 leader_election.cc:334] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365): Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
I20260808 00:42:48.304077 31336 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 707d92be9cbc45278e303eff39258662 in term 3.
I20260808 00:42:48.304311 31032 leader_election.cc:302] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 700e8c77909849b49ea4c79ccd7ac037, 707d92be9cbc45278e303eff39258662; no voters: 837aae531f9e489d97bb37425d5df64e
I20260808 00:42:48.304447 32074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:42:48.304601 32074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [term 3 LEADER]: Becoming Leader. State: Replica: 707d92be9cbc45278e303eff39258662, State: Running, Role: LEADER
I20260808 00:42:48.304701 32074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } }
I20260808 00:42:48.305169 32077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707d92be9cbc45278e303eff39258662. Latest consensus state: current_term: 3 leader_uuid: "707d92be9cbc45278e303eff39258662" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:48.305265 32077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:48.305483 32079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:48.306188 32079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:48.306315 32079 catalog_manager.cc:1259] Loaded cluster ID: 432c4cd1754d41b8bbadbafec7495d8d
I20260808 00:42:48.306381 32079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:48.306622 32079 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:48.306736 32079 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Loaded TSK: 0
I20260808 00:42:48.306938 32079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:42:48.353802 31051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:48.356176 31051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:48.357601 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.357448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49836)
0808 00:42:48.357516 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.357568 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 00:42:48.357650 31336 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [term 3 FOLLOWER]: Refusing update from remote peer 707d92be9cbc45278e303eff39258662: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 00:42:48.358011 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.357907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.0.0.1:49840)
0808 00:42:48.357956 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.357987 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:42:48.358067 32077 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [LEADER]: Connected to new peer: Peer: permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 00:42:48.358206 31032 consensus_peers.cc:487] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 -> Peer 837aae531f9e489d97bb37425d5df64e (127.20.105.126:44365): Couldn't send request to peer 837aae531f9e489d97bb37425d5df64e. Status: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 00:42:48.358990 32082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707d92be9cbc45278e303eff39258662. Latest consensus state: current_term: 3 leader_uuid: "707d92be9cbc45278e303eff39258662" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:48.359153 32082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:42:48.359608 32081 mvcc.cc:205] Tried to move back new op lower bound from 7316069451190108160 to 7316069450976956416. Current Snapshot: MvccSnapshot[applied={T|T < 7316069451190108160}]
I20260808 00:42:48.359748 32074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "707d92be9cbc45278e303eff39258662" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:48.359885 32083 mvcc.cc:205] Tried to move back new op lower bound from 7316069451190108160 to 7316069450976956416. Current Snapshot: MvccSnapshot[applied={T|T < 7316069451190108160}]
I20260808 00:42:48.359910 32074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:48.360110 32082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "707d92be9cbc45278e303eff39258662" committed_config { opid_index: 10 OBSOLETE_local: true peers { permanent_uuid: "837aae531f9e489d97bb37425d5df64e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44365 } } peers { permanent_uuid: "707d92be9cbc45278e303eff39258662" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 43651 } attrs { promote: false } } peers { permanent_uuid: "700e8c77909849b49ea4c79ccd7ac037" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 36103 } attrs { promote: false } } }
I20260808 00:42:48.360208 32082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 700e8c77909849b49ea4c79ccd7ac037 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:42:48.366026 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.362289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:40329 (local address 127.0.0.1:49356)
0808 00:42:48.362536 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.362550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:48.362715 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:48.363036 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:48.363037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:48.363040 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:48.363099 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:48.363101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.363988 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.363989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:48.364914 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:48.364916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.365738 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.365739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:48.365814 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:48.365821 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:42:48.365869 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:42:48.365918 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":99,"threads_started":1}
I20260808 00:42:48.366163 32075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.362178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:38339 (local address 127.0.0.1:48554)
0808 00:42:48.362230 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.362260 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:48.362337 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:48.362702 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:48.362703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:48.362705 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:48.362786 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:48.362788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.363854 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.363855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:48.365072 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:48.365075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.365862 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.365863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:48.365935 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:48.365941 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:42:48.366000 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:42:48.366053 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 00:42:48.366273 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.362499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:38391 (local address 127.0.0.1:56224)
0808 00:42:48.363078 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.363088 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:48.363145 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:48.363370 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:48.363371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:48.363385 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:48.363463 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:48.363465 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.364355 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.364357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:48.365377 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:48.365379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.366020 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.366021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:48.366102 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:48.366107 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:48.366166 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:42:48.366208 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":173,"threads_started":1}
I20260808 00:42:48.366351 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.362392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49356 (local address 127.20.105.67:40329)
0808 00:42:48.362561 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:48.362563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:48.362857 (+ 294us) server_negotiation.cc:407] Connection header received
0808 00:42:48.362897 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:48.362898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:48.362914 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:48.362982 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:48.363180 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.363910 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.365004 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.365616 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.365687 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:48.365703 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:48.366199 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 00:42:48.366256 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260808 00:42:48.366521 32090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.362292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48554 (local address 127.20.105.65:38339)
0808 00:42:48.362433 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:48.362435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:48.362443 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:48.362477 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:48.362478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:48.362494 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:48.362700 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:48.362886 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.363766 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.365168 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.365772 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.365847 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:48.365863 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:48.366368 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 00:42:48.366435 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260808 00:42:48.366727 31865 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842 (DEFAULT_TABLE table=test-table [id=7fc6b1822fe84eb3939f1827086c3a15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:48.366775 31477 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842 (DEFAULT_TABLE table=test-table [id=7fc6b1822fe84eb3939f1827086c3a15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:48.366915 31477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:48.366915 31865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:48.368824 32100 tablet_bootstrap.cc:492] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026: Bootstrap starting.
I20260808 00:42:48.369058 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.362925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.20.105.66:38391)
0808 00:42:48.363226 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:48.363228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:48.363235 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:42:48.363261 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:48.363262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:48.363270 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:48.363370 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:48.363553 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.364259 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.365471 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.365934 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.366011 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:48.366024 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:48.368868 (+ 2844us) server_negotiation.cc:299] Negotiation successful
0808 00:42:48.368931 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":84,"threads_started":1}
I20260808 00:42:48.369338 32100 tablet_bootstrap.cc:654] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:48.369349 31670 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842 (DEFAULT_TABLE table=test-table [id=7fc6b1822fe84eb3939f1827086c3a15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:48.369488 31670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:48.370123 32098 tablet_bootstrap.cc:492] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: Bootstrap starting.
I20260808 00:42:48.370615 32098 tablet_bootstrap.cc:654] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:48.371109 32100 tablet_bootstrap.cc:492] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026: No bootstrap required, opened a new log
I20260808 00:42:48.371173 32100 ts_tablet_manager.cc:1397] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:42:48.371369 32098 tablet_bootstrap.cc:492] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: No bootstrap required, opened a new log
I20260808 00:42:48.371443 32098 ts_tablet_manager.cc:1397] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:48.371389 32100 raft_consensus.cc:348] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.371487 32100 raft_consensus.cc:374] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:48.371515 32100 raft_consensus.cc:729] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a081ed75d561474bba9a154f25754026, State: Initialized, Role: FOLLOWER
I20260808 00:42:48.371585 32098 raft_consensus.cc:348] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.371605 32100 consensus_queue.cc:260] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.371667 32098 raft_consensus.cc:374] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:48.371701 32098 raft_consensus.cc:729] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43c44f6b5ab243b59df0aaa532bb86ff, State: Initialized, Role: FOLLOWER
I20260808 00:42:48.371763 32098 consensus_queue.cc:260] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.371863 32100 ts_tablet_manager.cc:1428] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:48.371980 32098 ts_tablet_manager.cc:1428] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:42:48.372131 32104 tablet_bootstrap.cc:492] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: Bootstrap starting.
I20260808 00:42:48.372442 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.372345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:40363)
0808 00:42:48.372388 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.372411 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 00:42:48.372442 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.372196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:59811)
0808 00:42:48.372363 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.372412 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 00:42:48.372556 32104 tablet_bootstrap.cc:654] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:48.373134 31571 heartbeater.cc:503] Master 127.20.105.125:43651 was elected leader, sending a full tablet report...
I20260808 00:42:48.373188 31956 heartbeater.cc:503] Master 127.20.105.125:43651 was elected leader, sending a full tablet report...
I20260808 00:42:48.373301 32104 tablet_bootstrap.cc:492] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: No bootstrap required, opened a new log
I20260808 00:42:48.373350 32104 ts_tablet_manager.cc:1397] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:48.373526 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.373431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:47311)
0808 00:42:48.373475 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.373502 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:42:48.373538 32104 raft_consensus.cc:348] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.373636 32104 raft_consensus.cc:374] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:48.373637 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.373229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:40049)
0808 00:42:48.373570 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.373605 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320}
I20260808 00:42:48.373658 32104 raft_consensus.cc:729] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be4b9e066454bb3ba060bfb7145e179, State: Initialized, Role: FOLLOWER
I20260808 00:42:48.373708 32104 consensus_queue.cc:260] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.373920 32104 ts_tablet_manager.cc:1428] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:48.374238 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.374101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:34209)
0808 00:42:48.374181 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.374209 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 00:42:48.374593 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.374517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:38485)
0808 00:42:48.374542 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.374574 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260808 00:42:48.403864 31957 tablet.cc:2369] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:48.665009 32107 raft_consensus.cc:482] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:48.665158 32107 raft_consensus.cc:504] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.665405 32107 leader_election.cc:288] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a081ed75d561474bba9a154f25754026 (127.20.105.65:38339), 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391)
I20260808 00:42:48.665822 31502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae5e460f1cb4c75900aeb8cbc9bc842" candidate_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a081ed75d561474bba9a154f25754026" is_pre_election: true
I20260808 00:42:48.665833 31703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae5e460f1cb4c75900aeb8cbc9bc842" candidate_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be4b9e066454bb3ba060bfb7145e179" is_pre_election: true
I20260808 00:42:48.665946 31502 raft_consensus.cc:2454] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43c44f6b5ab243b59df0aaa532bb86ff in term 0.
I20260808 00:42:48.665997 31703 raft_consensus.cc:2454] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43c44f6b5ab243b59df0aaa532bb86ff in term 0.
I20260808 00:42:48.666234 31798 leader_election.cc:302] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43c44f6b5ab243b59df0aaa532bb86ff, a081ed75d561474bba9a154f25754026; no voters:
I20260808 00:42:48.666356 32107 raft_consensus.cc:2781] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:48.666402 32107 raft_consensus.cc:482] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:48.666432 32107 raft_consensus.cc:3037] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:48.667410 32107 raft_consensus.cc:504] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.667587 32107 leader_election.cc:288] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [CANDIDATE]: Term 1 election: Requested vote from peers a081ed75d561474bba9a154f25754026 (127.20.105.65:38339), 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391)
I20260808 00:42:48.667812 31703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae5e460f1cb4c75900aeb8cbc9bc842" candidate_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5be4b9e066454bb3ba060bfb7145e179"
I20260808 00:42:48.667810 31502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae5e460f1cb4c75900aeb8cbc9bc842" candidate_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a081ed75d561474bba9a154f25754026"
I20260808 00:42:48.667927 31703 raft_consensus.cc:3037] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:48.667961 31502 raft_consensus.cc:3037] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:48.668982 31703 raft_consensus.cc:2454] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43c44f6b5ab243b59df0aaa532bb86ff in term 1.
I20260808 00:42:48.668982 31502 raft_consensus.cc:2454] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43c44f6b5ab243b59df0aaa532bb86ff in term 1.
I20260808 00:42:48.669204 31798 leader_election.cc:302] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43c44f6b5ab243b59df0aaa532bb86ff, a081ed75d561474bba9a154f25754026; no voters:
I20260808 00:42:48.669320 32107 raft_consensus.cc:2781] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:48.669530 32107 raft_consensus.cc:686] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 LEADER]: Becoming Leader. State: Replica: 43c44f6b5ab243b59df0aaa532bb86ff, State: Running, Role: LEADER
I20260808 00:42:48.669589 32107 consensus_queue.cc:237] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } }
I20260808 00:42:48.670145 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.669971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:39603)
0808 00:42:48.670033 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.670069 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 00:42:48.670658 31053 catalog_manager.cc:5495] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 43c44f6b5ab243b59df0aaa532bb86ff (127.20.105.67). New cstate: current_term: 1 leader_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43c44f6b5ab243b59df0aaa532bb86ff" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:48.723049 32095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.720520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49372 (local address 127.20.105.67:40329)
0808 00:42:48.720600 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:48.720603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:48.720612 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:42:48.720645 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:48.720645 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:48.720658 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:48.720714 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:48.720947 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.721546 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.722366 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:48.722597 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:48.722664 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:48.722775 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:48.722805 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:48.722840 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:48.722897 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 00:42:48.722916 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:42:48.724300 31703 raft_consensus.cc:1261] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 1 FOLLOWER]: Refusing update from remote peer 43c44f6b5ab243b59df0aaa532bb86ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:48.724320 31502 raft_consensus.cc:1261] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 1 FOLLOWER]: Refusing update from remote peer 43c44f6b5ab243b59df0aaa532bb86ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:48.724614 32114 consensus_queue.cc:1035] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "5be4b9e066454bb3ba060bfb7145e179" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:48.724774 32107 consensus_queue.cc:1035] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "a081ed75d561474bba9a154f25754026" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:48.725452 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.725288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:58463)
0808 00:42:48.725347 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.725391 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:42:48.725488 31770 heartbeater.cc:503] Master 127.20.105.125:43651 was elected leader, sending a full tablet report...
I20260808 00:42:48.726336 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.726154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:53421)
0808 00:42:48.726247 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.726292 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 00:42:48.726974 32118 mvcc.cc:205] Tried to move back new op lower bound from 7316069452692406272 to 7316069452471255040. Current Snapshot: MvccSnapshot[applied={T|T < 7316069452692406272}]
I20260808 00:42:48.727442 32119 mvcc.cc:205] Tried to move back new op lower bound from 7316069452692406272 to 7316069452471255040. Current Snapshot: MvccSnapshot[applied={T|T < 7316069452692406272}]
I20260808 00:42:48.727834 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.727682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:35253)
0808 00:42:48.727768 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.727798 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 00:42:48.728304 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.728200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.66:47007)
0808 00:42:48.728238 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.728269 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 00:42:48.729136 32070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.729015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.67:56819)
0808 00:42:48.729073 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.729108 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 00:42:48.729350 31051 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:48.729472 31051 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:48.729637 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.729541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44365 (local address 127.20.105.65:36009)
0808 00:42:48.729580 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.729605 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:44365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 00:42:48.731120 31051 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 707d92be9cbc45278e303eff39258662: Sending DeleteTablet for 3 replicas of tablet 2ae5e460f1cb4c75900aeb8cbc9bc842
I20260808 00:42:48.731452 31477 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:48 UTC) from {username='oryx', principal='oryx/127.20.105.125@KRBTEST.COM'} at 127.0.0.1:48554
I20260808 00:42:48.731526 31670 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:48 UTC) from {username='oryx', principal='oryx/127.20.105.125@KRBTEST.COM'} at 127.0.0.1:56224
I20260808 00:42:48.731701 32132 tablet_replica.cc:330] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026: stopping tablet replica
I20260808 00:42:48.731753 32133 tablet_replica.cc:330] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: stopping tablet replica
I20260808 00:42:48.731776 32132 raft_consensus.cc:2229] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:48.731805 32133 raft_consensus.cc:2229] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:48.731827 32132 raft_consensus.cc:2258] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:48.731849 32133 raft_consensus.cc:2258] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:48.731984 31865 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:48 UTC) from {username='oryx', principal='oryx/127.20.105.125@KRBTEST.COM'} at 127.0.0.1:49356
I20260808 00:42:48.732197 32133 ts_tablet_manager.cc:1905] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:48.732218 32134 tablet_replica.cc:330] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: stopping tablet replica
I20260808 00:42:48.732275 32134 raft_consensus.cc:2229] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:48.732276 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 31271
I20260808 00:42:48.732337 32134 raft_consensus.cc:2258] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:48.732436 32132 ts_tablet_manager.cc:1905] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P a081ed75d561474bba9a154f25754026: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:48.733289 32134 ts_tablet_manager.cc:1905] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:48.734751 32133 ts_tablet_manager.cc:1918] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:48.734818 32133 log.cc:1196] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/2ae5e460f1cb4c75900aeb8cbc9bc842
I20260808 00:42:48.735080 32133 ts_tablet_manager.cc:1939] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 5be4b9e066454bb3ba060bfb7145e179: Deleting consensus metadata
I20260808 00:42:48.735452 32134 ts_tablet_manager.cc:1918] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:48.735517 32134 log.cc:1196] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/2ae5e460f1cb4c75900aeb8cbc9bc842
I20260808 00:42:48.735776 32134 ts_tablet_manager.cc:1939] T 2ae5e460f1cb4c75900aeb8cbc9bc842 P 43c44f6b5ab243b59df0aaa532bb86ff: Deleting consensus metadata
I20260808 00:42:48.744773 31028 catalog_manager.cc:4841] TS 43c44f6b5ab243b59df0aaa532bb86ff (127.20.105.67:40329): tablet 2ae5e460f1cb4c75900aeb8cbc9bc842 (table test-table [id=7fc6b1822fe84eb3939f1827086c3a15]) successfully deleted
I20260808 00:42:48.744942 31028 catalog_manager.cc:4841] TS 5be4b9e066454bb3ba060bfb7145e179 (127.20.105.66:38391): tablet 2ae5e460f1cb4c75900aeb8cbc9bc842 (table test-table [id=7fc6b1822fe84eb3939f1827086c3a15]) successfully deleted
I20260808 00:42:48.753470 32096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:48.753287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:38339 (local address 127.0.0.1:48568)
0808 00:42:48.753364 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:48.753409 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:38339: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 00:42:48.753688 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 31582
W20260808 00:42:48.753743 31028 catalog_manager.cc:4584] TS a081ed75d561474bba9a154f25754026 (127.20.105.65:38339): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ae5e460f1cb4c75900aeb8cbc9bc842: Network error: Client connection negotiation failed: client connection to 127.20.105.65:38339: connect: Connection refused (error 111)
I20260808 00:42:48.772416 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 31775
I20260808 00:42:48.790164 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 30854
I20260808 00:42:48.806265 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 31132
2026-08-08T00:42:48Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7601 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:48 dist-test-slave-kvfs krb5kdc[32137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:48 dist-test-slave-kvfs krb5kdc[32137](info): set up 2 sockets
Aug 08 00:42:48 dist-test-slave-kvfs krb5kdc[32137](info): commencing operation
krb5kdc: starting...
W20260808 00:42:50.132828 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.284s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:50 dist-test-slave-kvfs krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.126.keytab.
Entry for principal oryx/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.126.keytab.
I20260808 00:42:50.288374 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44659
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.126.keytab
--principal=oryx/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:44659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:50.417383 32160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:50.417889 32160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:50.418062 32160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:50.420451 32160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:50.420600 32160 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:42:50.420656 32160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:50.420745 32160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:50.420822 32160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:50.420881 32160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:50.424430 32160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:44659
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.126.keytab
--principal=oryx/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:50.425033 32160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:50.425557 32160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:50.435195 32175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:50.435189 32173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:50.435189 32171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:50.435421 32160 server_base.cc:1034] running on GCE node
I20260808 00:42:50.436168 32160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:50.436729 32160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:50.437974 32160 hybrid_clock.cc:648] HybridClock initialized: now 1786149770437955 us; error 76 us; skew 500 ppm
Aug 08 00:42:50 dist-test-slave-kvfs krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149770, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:50.442728 32160 init.cc:377] Logged in from keytab as oryx/127.20.105.126@KRBTEST.COM (short username oryx)
I20260808 00:42:50.443667 32160 webserver.cc:460] Webserver started at http://127.20.105.126:38989/ using document root <none> and password file <none>
I20260808 00:42:50.444160 32160 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:50.444262 32160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:50.444473 32160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:50.446496 32160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "85e7de31fd01456e9037e9709ac2656a"
format_stamp: "Formatted at 2026-08-08 00:42:50 on dist-test-slave-kvfs"
server_key: "4ef151b7705e2f3615c1ea284fe83e24"
server_key_iv: "533fcf6a895e7b2d7e906f266f6e7079"
server_key_version: "encryptionkey@0"
I20260808 00:42:50.447108 32160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "85e7de31fd01456e9037e9709ac2656a"
format_stamp: "Formatted at 2026-08-08 00:42:50 on dist-test-slave-kvfs"
server_key: "4ef151b7705e2f3615c1ea284fe83e24"
server_key_iv: "533fcf6a895e7b2d7e906f266f6e7079"
server_key_version: "encryptionkey@0"
I20260808 00:42:50.449973 32160 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 00:42:50.451409 32185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:50.451854 32160 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:42:50.452116 32160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "85e7de31fd01456e9037e9709ac2656a"
format_stamp: "Formatted at 2026-08-08 00:42:50 on dist-test-slave-kvfs"
server_key: "4ef151b7705e2f3615c1ea284fe83e24"
server_key_iv: "533fcf6a895e7b2d7e906f266f6e7079"
server_key_version: "encryptionkey@0"
I20260808 00:42:50.452245 32160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:50.475519 32160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:50.480751 32160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:50.481148 32160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:50.492925 32160 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:44659
I20260808 00:42:50.493053 32271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:44659 every 8 connection(s)
I20260808 00:42:50.493811 32160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:50.495249 32273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:50.498783 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 32160
I20260808 00:42:50.498991 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:50.499400 20901 external_mini_cluster.cc:1428] Setting key 64db7b9d5a74051c3febc00265c2140e
I20260808 00:42:50.500988 32273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: Bootstrap starting.
I20260808 00:42:50.502214 32273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:50.502807 32273 log.cc:824] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:50.504547 32273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: No bootstrap required, opened a new log
I20260808 00:42:50.509483 32273 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e7de31fd01456e9037e9709ac2656a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44659 } }
I20260808 00:42:50.509822 32273 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:50.509946 32273 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e7de31fd01456e9037e9709ac2656a, State: Initialized, Role: FOLLOWER
I20260808 00:42:50.510344 32273 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e7de31fd01456e9037e9709ac2656a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44659 } }
I20260808 00:42:50.510541 32273 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:50.510622 32273 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:50.510757 32273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:50.512476 32273 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e7de31fd01456e9037e9709ac2656a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44659 } }
I20260808 00:42:50.512733 32273 leader_election.cc:302] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85e7de31fd01456e9037e9709ac2656a; no voters:
I20260808 00:42:50.513042 32273 leader_election.cc:288] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:50.513127 32281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:50.513474 32281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [term 1 LEADER]: Becoming Leader. State: Replica: 85e7de31fd01456e9037e9709ac2656a, State: Running, Role: LEADER
I20260808 00:42:50.513631 32273 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:50.513808 32281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e7de31fd01456e9037e9709ac2656a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44659 } }
Aug 08 00:42:50 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:50.514817 32283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85e7de31fd01456e9037e9709ac2656a. Latest consensus state: current_term: 1 leader_uuid: "85e7de31fd01456e9037e9709ac2656a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e7de31fd01456e9037e9709ac2656a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44659 } } }
I20260808 00:42:50.514961 32283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:50.516101 32294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:50.516352 32284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85e7de31fd01456e9037e9709ac2656a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85e7de31fd01456e9037e9709ac2656a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44659 } } }
I20260808 00:42:50.516484 32284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:50.517235 32294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:50.522496 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:50.500550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.20.105.126:44659)
0808 00:42:50.500913 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:50.500940 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:50.500983 (+ 43us) server_negotiation.cc:407] Connection header received
0808 00:42:50.501865 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:50.501875 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:50.501980 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:50.502293 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:50.503698 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.506414 (+ 2716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.507747 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.509144 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.509404 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:50.515434 (+ 6030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:50.515439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:50.515440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:50.515478 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:50.519277 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:50.520240 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:50.520254 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:50.520254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:50.520417 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:50.520834 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:50.520835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:50.520836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:50.521276 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:50.521598 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:50.521851 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 00:42:50.522163 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":165,"threads_started":1}
I20260808 00:42:50.523332 32294 catalog_manager.cc:1347] Generated new cluster ID: 708d972ae58d4afab475bca14e47e0b1
I20260808 00:42:50.523424 32294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:50.541216 32294 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:50.542260 32294 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:50.549165 32294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: Generated new TSK 0
I20260808 00:42:50.549551 32294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab.
Entry for principal oryx/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab.
I20260808 00:42:50.609350 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44659
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab
--principal=oryx/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:50.789511 32308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:50.790145 32308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:50.790309 32308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:50.792889 32308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:50.793008 32308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:50.793115 32308 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:50.793231 32308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:50.796208 32308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab
--principal=oryx/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:50.796830 32308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:50.797327 32308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:50.808526 32323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:50.808526 32320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:50.808526 32319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:50.808561 32308 server_base.cc:1034] running on GCE node
I20260808 00:42:50.809439 32308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:50.809919 32308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:50.811154 32308 hybrid_clock.cc:648] HybridClock initialized: now 1786149770811109 us; error 68 us; skew 500 ppm
Aug 08 00:42:50 dist-test-slave-kvfs krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149770, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:50.816787 32308 init.cc:377] Logged in from keytab as oryx/127.20.105.65@KRBTEST.COM (short username oryx)
I20260808 00:42:50.817981 32308 webserver.cc:460] Webserver started at http://127.20.105.65:35267/ using document root <none> and password file <none>
I20260808 00:42:50.818462 32308 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:50.818584 32308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:50.818869 32308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:50.821203 32308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "eb35618b1ef44514ab63594cc2921b4a"
format_stamp: "Formatted at 2026-08-08 00:42:50 on dist-test-slave-kvfs"
server_key: "51912ae2824294f871b890c503e8e6cb"
server_key_iv: "f8fd1d667c1c7af00ebe772467eff22d"
server_key_version: "encryptionkey@0"
I20260808 00:42:50.822039 32308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "eb35618b1ef44514ab63594cc2921b4a"
format_stamp: "Formatted at 2026-08-08 00:42:50 on dist-test-slave-kvfs"
server_key: "51912ae2824294f871b890c503e8e6cb"
server_key_iv: "f8fd1d667c1c7af00ebe772467eff22d"
server_key_version: "encryptionkey@0"
I20260808 00:42:50.825695 32308 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 00:42:50.827419 32334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:50.827864 32308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:50.828150 32308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "eb35618b1ef44514ab63594cc2921b4a"
format_stamp: "Formatted at 2026-08-08 00:42:50 on dist-test-slave-kvfs"
server_key: "51912ae2824294f871b890c503e8e6cb"
server_key_iv: "f8fd1d667c1c7af00ebe772467eff22d"
server_key_version: "encryptionkey@0"
I20260808 00:42:50.828332 32308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:50.862735 32308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:50.868340 32308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:50.868726 32308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:50.870055 32308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:50.870157 32308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:50.870306 32308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:50.870388 32308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:42:50 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149770, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:50.889096 32308 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:46127
I20260808 00:42:50.889211 32499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:46127 every 8 connection(s)
I20260808 00:42:50.889618 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:50.870923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:38469 (local address 127.20.105.126:44659)
0808 00:42:50.871048 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:50.871051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:50.871524 (+ 473us) server_negotiation.cc:407] Connection header received
0808 00:42:50.872655 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:50.872656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:50.872690 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:50.872862 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:50.874236 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.875828 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.877846 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.878617 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.878783 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:50.883801 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:50.883804 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:50.883804 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:50.883824 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:50.886587 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:50.887448 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:50.887460 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:50.887461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:50.887559 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:50.888082 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:50.888091 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:50.888092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:50.888334 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:50.888491 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:50.889247 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 00:42:50.889499 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 00:42:50.890053 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:50.870959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44659 (local address 127.20.105.65:38469)
0808 00:42:50.871246 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:50.871337 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:50.872439 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:50.873017 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:50.873030 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:50.873551 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:50.874071 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:50.874079 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.876130 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.876131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:50.877676 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:50.877679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.878793 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.878794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:50.879063 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:50.880145 (+ 1082us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:50.880186 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:50.883589 (+ 3403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:50.886783 (+ 3194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:50.886793 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:50.886797 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:50.887287 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:50.887692 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:50.887693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:50.887693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:50.887909 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:50.888461 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:50.888478 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:50.888865 (+ 387us) client_negotiation.cc:769] Sending connection context
0808 00:42:50.889130 (+ 265us) client_negotiation.cc:240] Negotiation successful
0808 00:42:50.889536 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":145,"threads_started":1}
I20260808 00:42:50.890079 32308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:50.891875 32500 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44659
I20260808 00:42:50.892120 32500 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:50.892601 32500 heartbeater.cc:511] Master 127.20.105.126:44659 requested a full tablet report, sending...
I20260808 00:42:50.893944 32206 ts_manager.cc:194] Registered new tserver with Master: eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127)
I20260808 00:42:50.895447 32206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:38469
I20260808 00:42:50.895547 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 32308
I20260808 00:42:50.895727 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:50.896157 20901 external_mini_cluster.cc:1428] Setting key 7bbb00c8a868bed25b92baef29c2cce1
I20260808 00:42:50.905407 32509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:50.893411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44659 (local address 127.20.105.65:47607)
0808 00:42:50.893695 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:50.893719 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:50.893825 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:50.894064 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:50.894066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:50.894480 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:50.894616 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:50.894619 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.896143 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.896144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:50.897204 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:50.897217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.897849 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.897850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:50.897963 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:50.898785 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:50.898792 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:50.899623 (+ 831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:50.903488 (+ 3865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:50.903490 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:50.903491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:50.903805 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:50.904245 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:50.904246 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:50.904246 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:50.904343 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:50.904915 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:50.904916 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:50.904997 (+ 81us) client_negotiation.cc:769] Sending connection context
0808 00:42:50.905102 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 00:42:50.905269 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":148,"threads_started":1}
I20260808 00:42:50.905660 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:50.893490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:47607 (local address 127.20.105.126:44659)
0808 00:42:50.893593 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:50.893595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:50.893816 (+ 221us) server_negotiation.cc:407] Connection header received
0808 00:42:50.893932 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:50.893940 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:50.893952 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:50.894037 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:50.894784 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.895658 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.897346 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:50.897745 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:50.897845 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:50.899754 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:50.899755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:50.899755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:50.899763 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:50.903303 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:50.903929 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:50.903931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:50.903932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:50.904120 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:50.904449 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:50.904450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:50.904451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:50.904812 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:50.904937 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:50.905183 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 00:42:50.905383 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for oryx/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab.
Entry for principal oryx/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab.
I20260808 00:42:50.949594 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44659
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab
--principal=oryx/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:51.115717 32512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:51.116509 32512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:51.116604 32512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:51.120914 32512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:51.121075 32512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:51.121199 32512 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:51.121299 32512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:51.127197 32512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab
--principal=oryx/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:51.132097 32512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:51.132896 32512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:51.144143 32521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:51.144143 32524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:51.144344 32520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:51.144830 32512 server_base.cc:1034] running on GCE node
I20260808 00:42:51.145249 32512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:51.145705 32512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:51.146914 32512 hybrid_clock.cc:648] HybridClock initialized: now 1786149771146899 us; error 67 us; skew 500 ppm
Aug 08 00:42:51 dist-test-slave-kvfs krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149771, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:51.152349 32512 init.cc:377] Logged in from keytab as oryx/127.20.105.66@KRBTEST.COM (short username oryx)
I20260808 00:42:51.153360 32512 webserver.cc:460] Webserver started at http://127.20.105.66:44165/ using document root <none> and password file <none>
I20260808 00:42:51.153825 32512 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:51.153949 32512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:51.154217 32512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:51.156453 32512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "9d13faf37abd4b0c8a108fee6854e0e4"
format_stamp: "Formatted at 2026-08-08 00:42:51 on dist-test-slave-kvfs"
server_key: "5e056375a5cde3e1f54c821e0b8fc9a0"
server_key_iv: "b6bc039b87963d51c9afd4c04cecdd09"
server_key_version: "encryptionkey@0"
I20260808 00:42:51.157214 32512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "9d13faf37abd4b0c8a108fee6854e0e4"
format_stamp: "Formatted at 2026-08-08 00:42:51 on dist-test-slave-kvfs"
server_key: "5e056375a5cde3e1f54c821e0b8fc9a0"
server_key_iv: "b6bc039b87963d51c9afd4c04cecdd09"
server_key_version: "encryptionkey@0"
I20260808 00:42:51.160772 32512 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 00:42:51.162559 32533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:51.162945 32512 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:51.163149 32512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "9d13faf37abd4b0c8a108fee6854e0e4"
format_stamp: "Formatted at 2026-08-08 00:42:51 on dist-test-slave-kvfs"
server_key: "5e056375a5cde3e1f54c821e0b8fc9a0"
server_key_iv: "b6bc039b87963d51c9afd4c04cecdd09"
server_key_version: "encryptionkey@0"
I20260808 00:42:51.163300 32512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:51.185998 32512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:51.190448 32512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:51.190759 32512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:51.191985 32512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:51.192132 32512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:51.192235 32512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:51.192286 32512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:42:51 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149771, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:51.208312 32512 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:33409
I20260808 00:42:51.208501 32707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:33409 every 8 connection(s)
I20260808 00:42:51.209173 32512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:51.211542 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.192869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:35525 (local address 127.20.105.126:44659)
0808 00:42:51.192981 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.192984 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.193376 (+ 392us) server_negotiation.cc:407] Connection header received
0808 00:42:51.194394 (+ 1018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.194396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.194431 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.194610 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:51.195857 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.196746 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.198640 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.198955 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.199107 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.204456 (+ 5349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:51.204459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:51.204459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:51.204486 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:51.208574 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.209553 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.209555 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.209555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.209702 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.210305 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.210306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.210306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.210644 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:51.210765 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.211215 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.211379 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:42:51.211849 32544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.192857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44659 (local address 127.20.105.66:35525)
0808 00:42:51.193141 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.193215 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.194216 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.194726 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.194738 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.195231 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:51.195687 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.195694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.196934 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.196935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.198456 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.198459 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.199137 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.199139 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.199430 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.200567 (+ 1137us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:51.200577 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:51.204250 (+ 3673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:51.208842 (+ 4592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:51.208848 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:51.208850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:51.209383 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:51.209877 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:51.209878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:51.209879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:51.210177 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:51.210741 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:51.210755 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:51.210989 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.211173 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.211427 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":162,"threads_started":1}
I20260808 00:42:51.212831 32710 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44659
I20260808 00:42:51.213032 32710 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:51.213595 32710 heartbeater.cc:511] Master 127.20.105.126:44659 requested a full tablet report, sending...
I20260808 00:42:51.214319 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 32512
I20260808 00:42:51.214625 32206 ts_manager.cc:194] Registered new tserver with Master: 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409)
I20260808 00:42:51.214641 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:51.215108 20901 external_mini_cluster.cc:1428] Setting key 742f495f8fe7c9cbdf66a83421a5e38a
I20260808 00:42:51.215400 32206 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:35525
I20260808 00:42:51.226406 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.215386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:40513 (local address 127.20.105.126:44659)
0808 00:42:51.215476 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.215478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.215486 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:51.215531 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.215532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.215545 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.215661 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:51.216443 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.217518 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.218791 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.219251 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.219404 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.221377 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:51.221379 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:51.221379 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:51.221396 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:51.224121 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.224976 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.224977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.224978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.225075 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.225472 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.225473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.225473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.225732 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:51.225817 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.226158 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.226280 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 00:42:51.226678 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.214988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44659 (local address 127.20.105.66:40513)
0808 00:42:51.215208 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.215221 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.215286 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.215720 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.215722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.216178 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:51.216285 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.216291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.217734 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.217735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.218666 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.218669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.219387 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.219387 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.219496 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.220602 (+ 1106us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:51.220607 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:51.221195 (+ 588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:51.224258 (+ 3063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:51.224260 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:51.224261 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:51.224798 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:51.225183 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:51.225184 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:51.225185 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:51.225337 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:51.225837 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:51.225839 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:51.225954 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.226119 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.226414 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":149,"threads_started":1}
WARNING: no policy specified for oryx/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab.
Entry for principal oryx/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab.
I20260808 00:42:51.272879 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44659
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab
--principal=oryx/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:51.442157 32721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:51.442876 32721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:51.442973 32721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:51.445724 32721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:51.445902 32721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:51.446028 32721 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:42:51.446126 32721 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:51.450165 32721 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab
--principal=oryx/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.105.126:44659
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:51.450802 32721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:51.451370 32721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:51.461756 32726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:51.461756 32727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:51.461757 32729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:51.462718 32721 server_base.cc:1034] running on GCE node
I20260808 00:42:51.463165 32721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:51.463764 32721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:51.465054 32721 hybrid_clock.cc:648] HybridClock initialized: now 1786149771465023 us; error 102 us; skew 500 ppm
Aug 08 00:42:51 dist-test-slave-kvfs krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149771, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:51.469566 32721 init.cc:377] Logged in from keytab as oryx/127.20.105.67@KRBTEST.COM (short username oryx)
I20260808 00:42:51.470268 32721 webserver.cc:460] Webserver started at http://127.20.105.67:33853/ using document root <none> and password file <none>
I20260808 00:42:51.470781 32721 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:51.470914 32721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:51.471275 32721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:51.473839 32721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "264eb8e4d312471993bb5b9f89e9722f"
format_stamp: "Formatted at 2026-08-08 00:42:51 on dist-test-slave-kvfs"
server_key: "3357dc2f13058db25dc3c03c503a1057"
server_key_iv: "fedb14abef6a110319921a6f3ebea6c4"
server_key_version: "encryptionkey@0"
I20260808 00:42:51.474658 32721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "264eb8e4d312471993bb5b9f89e9722f"
format_stamp: "Formatted at 2026-08-08 00:42:51 on dist-test-slave-kvfs"
server_key: "3357dc2f13058db25dc3c03c503a1057"
server_key_iv: "fedb14abef6a110319921a6f3ebea6c4"
server_key_version: "encryptionkey@0"
I20260808 00:42:51.477747 32721 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 00:42:51.479426 32738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:51.479768 32721 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:51.479982 32721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "264eb8e4d312471993bb5b9f89e9722f"
format_stamp: "Formatted at 2026-08-08 00:42:51 on dist-test-slave-kvfs"
server_key: "3357dc2f13058db25dc3c03c503a1057"
server_key_iv: "fedb14abef6a110319921a6f3ebea6c4"
server_key_version: "encryptionkey@0"
I20260808 00:42:51.480230 32721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:51.539333 32721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:51.543872 32721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:51.544312 32721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:51.545485 32721 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:51.545593 32721 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:51.545707 32721 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:51.545765 32721 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:42:51 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149771, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:51.561756 32721 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:38925
I20260808 00:42:51.561861 450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:38925 every 8 connection(s)
I20260808 00:42:51.562630 32721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:51.566147 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.546439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:52963 (local address 127.20.105.126:44659)
0808 00:42:51.546563 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.546566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.547077 (+ 511us) server_negotiation.cc:407] Connection header received
0808 00:42:51.548277 (+ 1200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.548279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.548296 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.548367 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:51.550175 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.550999 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.553211 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.553639 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.553802 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.559622 (+ 5820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:51.559624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:51.559625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:51.559644 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:51.562885 (+ 3241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.563840 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.563841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.563842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.563942 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.564553 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.564554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.564555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.564819 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:51.565000 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.565674 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.565895 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:42:51.566589 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.546459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44659 (local address 127.20.105.67:52963)
0808 00:42:51.546758 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.546889 (+ 131us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.548134 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.548487 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.548493 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.549296 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:51.549955 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.549973 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.551207 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.551209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.553033 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.553037 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.553823 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.553826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.554238 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.555461 (+ 1223us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:51.555489 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:51.559395 (+ 3906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:51.563082 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:51.563108 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:51.563112 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:51.563662 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:51.564125 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:51.564135 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:51.564136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:51.564389 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:51.564977 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:51.564995 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:51.565298 (+ 303us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.565576 (+ 278us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.566061 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":143,"threads_started":1}
I20260808 00:42:51.567701 452 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44659
I20260808 00:42:51.567914 452 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:51.568202 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 32721
I20260808 00:42:51.568459 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:51.568461 452 heartbeater.cc:511] Master 127.20.105.126:44659 requested a full tablet report, sending...
I20260808 00:42:51.568959 20901 external_mini_cluster.cc:1428] Setting key 197df605392fa79877e9ea167a103a7d
I20260808 00:42:51.569556 32205 ts_manager.cc:194] Registered new tserver with Master: 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:51.570466 32205 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:52963
I20260808 00:42:51.570688 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:42:51.582474 466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.570198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44659 (local address 127.20.105.67:34877)
0808 00:42:51.570419 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.570430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.570622 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.570915 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.570917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.571491 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:51.571637 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.571640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.572868 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.572870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.573759 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.573762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.574435 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.574436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.574576 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.575617 (+ 1041us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:51.575632 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:51.576414 (+ 782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:51.580553 (+ 4139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:51.580554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:51.580554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:51.580910 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:51.581263 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:51.581271 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:51.581272 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:51.581363 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:51.581928 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:51.581930 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:51.582022 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.582134 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.582318 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":128,"threads_started":1}
I20260808 00:42:51.582705 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.570226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:34877 (local address 127.20.105.126:44659)
0808 00:42:51.570305 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.570307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.570544 (+ 237us) server_negotiation.cc:407] Connection header received
0808 00:42:51.570725 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.570727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.570743 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.570908 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:51.571790 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.572675 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.573938 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.574262 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.574440 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.576636 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:51.576639 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:51.576639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:51.576673 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:51.580354 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.581027 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.581029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.581029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.581162 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.581446 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.581447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.581447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.581761 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:51.581950 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.582195 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.582401 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 00:42:51 dist-test-slave-kvfs krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149771, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:42:51 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:51 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:42:51.620591 32278 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:51.608595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34912 (local address 127.20.105.126:44659)
0808 00:42:51.608743 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.608757 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.608878 (+ 121us) server_negotiation.cc:407] Connection header received
0808 00:42:51.609101 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.609103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.609128 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.609312 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:51.610630 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.611575 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.613228 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.613725 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.613933 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.620339 (+ 6406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34912: BlockingRecv error: recv got EOF from 127.0.0.1:34912 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 00:42:51 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149771, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:51.636566 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.622273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34920 (local address 127.20.105.126:44659)
0808 00:42:51.622391 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.622394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.622593 (+ 199us) server_negotiation.cc:407] Connection header received
0808 00:42:51.622732 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.622733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.622754 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.622842 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:51.623776 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.624710 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.625989 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.626316 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.626504 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.631203 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:51.631206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:51.631206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:51.631236 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:51.634053 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.634795 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.634806 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.634807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.634938 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:51.635394 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:51.635405 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:51.635405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:51.635634 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:51.635889 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.636114 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.636295 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 00:42:51.639572 32205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:51.643234 32205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:51.657570 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.648817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46127 (local address 127.0.0.1:60528)
0808 00:42:51.649590 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.649680 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.649847 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.650687 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.650688 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.650689 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:51.650930 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.650934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.652669 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.652670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.654233 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.654236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.657072 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.657074 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.657218 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.657225 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.657289 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.657391 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":147,"thread_start_us":297,"threads_started":1}
I20260808 00:42:51.657707 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.649523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:33409 (local address 127.0.0.1:37432)
0808 00:42:51.650168 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.650180 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.650280 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.652107 (+ 1827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.652108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.652111 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:51.652262 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.652264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.654431 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.654432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.655893 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.655897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.657245 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.657246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.657351 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.657354 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.657439 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.657495 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":199,"threads_started":1}
I20260808 00:42:51.657771 494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.649105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37508 (local address 127.20.105.67:38925)
0808 00:42:51.649523 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.649529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.649921 (+ 392us) server_negotiation.cc:407] Connection header received
0808 00:42:51.650203 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.650207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.650245 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.650403 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:51.651081 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.652399 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.654866 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.656681 (+ 1815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.656827 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.657007 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.657470 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.657567 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":229,"threads_started":1}
I20260808 00:42:51.658254 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.649103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60528 (local address 127.20.105.65:46127)
0808 00:42:51.649580 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.649587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.649790 (+ 203us) server_negotiation.cc:407] Connection header received
0808 00:42:51.650475 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.650480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.650516 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.650685 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:51.651141 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.652477 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.654475 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.656681 (+ 2206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.656832 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.656961 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.658042 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.658130 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":204,"threads_started":1}
I20260808 00:42:51.658509 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.650239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37432 (local address 127.20.105.66:33409)
0808 00:42:51.650698 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.650703 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.650726 (+ 23us) server_negotiation.cc:407] Connection header received
0808 00:42:51.650821 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.650825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.650857 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.651043 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:51.652443 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.654292 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.656078 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.657079 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.657200 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.657349 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.657481 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.657571 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":192,"threads_started":1}
I20260808 00:42:51.658998 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.648816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:38925 (local address 127.0.0.1:37508)
0808 00:42:51.649775 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.649798 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.649930 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.650655 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.650670 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.650673 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:51.650930 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.650934 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.652557 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.652558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.654675 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.654677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.656873 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.656875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.657038 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.657080 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.657202 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.657302 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":91,"thread_start_us":264,"threads_started":1}
I20260808 00:42:51.659745 32610 tablet_service.cc:1467] Processing CreateTablet for tablet 672c859e2afb45c8a99d550cebafcb15 (DEFAULT_TABLE table=test-table [id=44e8ff6949d249acb14d9354174a0507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:51.659770 32406 tablet_service.cc:1467] Processing CreateTablet for tablet 672c859e2afb45c8a99d550cebafcb15 (DEFAULT_TABLE table=test-table [id=44e8ff6949d249acb14d9354174a0507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:51.659924 337 tablet_service.cc:1467] Processing CreateTablet for tablet 672c859e2afb45c8a99d550cebafcb15 (DEFAULT_TABLE table=test-table [id=44e8ff6949d249acb14d9354174a0507]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:51.660292 32406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 672c859e2afb45c8a99d550cebafcb15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:51.660292 32610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 672c859e2afb45c8a99d550cebafcb15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:51.660300 337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 672c859e2afb45c8a99d550cebafcb15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:51.664574 497 tablet_bootstrap.cc:492] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Bootstrap starting.
I20260808 00:42:51.665436 498 tablet_bootstrap.cc:492] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Bootstrap starting.
I20260808 00:42:51.666221 500 tablet_bootstrap.cc:492] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Bootstrap starting.
I20260808 00:42:51.666429 497 tablet_bootstrap.cc:654] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:51.667158 498 tablet_bootstrap.cc:654] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:51.667327 497 log.cc:824] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:51.667623 500 tablet_bootstrap.cc:654] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:51.667950 498 log.cc:824] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:51.668449 500 log.cc:824] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:51.669317 497 tablet_bootstrap.cc:492] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: No bootstrap required, opened a new log
I20260808 00:42:51.669467 497 ts_tablet_manager.cc:1397] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 00:42:51.669606 498 tablet_bootstrap.cc:492] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: No bootstrap required, opened a new log
I20260808 00:42:51.669752 498 ts_tablet_manager.cc:1397] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:42:51.670221 500 tablet_bootstrap.cc:492] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: No bootstrap required, opened a new log
I20260808 00:42:51.670362 500 ts_tablet_manager.cc:1397] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260808 00:42:51.674052 497 raft_consensus.cc:348] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.674441 497 raft_consensus.cc:374] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:51.674192 498 raft_consensus.cc:348] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.674563 497 raft_consensus.cc:729] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb35618b1ef44514ab63594cc2921b4a, State: Initialized, Role: FOLLOWER
I20260808 00:42:51.674602 498 raft_consensus.cc:374] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:51.674690 498 raft_consensus.cc:729] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d13faf37abd4b0c8a108fee6854e0e4, State: Initialized, Role: FOLLOWER
I20260808 00:42:51.674670 500 raft_consensus.cc:348] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.674985 500 raft_consensus.cc:374] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:51.674937 498 consensus_queue.cc:260] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.675057 500 raft_consensus.cc:729] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264eb8e4d312471993bb5b9f89e9722f, State: Initialized, Role: FOLLOWER
I20260808 00:42:51.674938 497 consensus_queue.cc:260] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.675345 500 consensus_queue.cc:260] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.675563 497 ts_tablet_manager.cc:1428] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260808 00:42:51.675563 498 ts_tablet_manager.cc:1428] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 00:42:51.675621 32500 heartbeater.cc:503] Master 127.20.105.126:44659 was elected leader, sending a full tablet report...
I20260808 00:42:51.675765 32710 heartbeater.cc:503] Master 127.20.105.126:44659 was elected leader, sending a full tablet report...
I20260808 00:42:51.676354 500 ts_tablet_manager.cc:1428] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 00:42:51.676611 452 heartbeater.cc:503] Master 127.20.105.126:44659 was elected leader, sending a full tablet report...
W20260808 00:42:51.710611 32711 tablet.cc:2369] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:42:51.814049 453 tablet.cc:2369] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:42:51.892175 32501 tablet.cc:2369] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:42:51.983769 507 raft_consensus.cc:482] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:51.983966 507 raft_consensus.cc:504] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.984705 507 leader_election.cc:288] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127), 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409)
I20260808 00:42:51.986552 504 raft_consensus.cc:482] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:51.986712 504 raft_consensus.cc:504] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.987747 504 leader_election.cc:288] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925), 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409)
I20260808 00:42:51.988883 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.985170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:33409 (local address 127.20.105.67:38773)
0808 00:42:51.985334 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.985347 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.985407 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.985586 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.985587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.985589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:51.985677 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.985678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.986476 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.986477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.987635 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.987639 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.988571 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.988572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.988662 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.988667 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.988719 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.988769 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":1,"thread_start_us":94,"threads_started":1}
I20260808 00:42:51.988941 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.985113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:39679 (local address 127.20.105.65:46127)
0808 00:42:51.985185 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.985187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.985199 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:51.985246 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.985252 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.985282 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.985367 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:51.985600 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.986265 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.987864 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.988504 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.988564 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.988573 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.988807 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.988852 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:42:51.989399 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.985389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:38773 (local address 127.20.105.66:33409)
0808 00:42:51.985445 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.985449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.985460 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:51.985501 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.985502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.985513 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.985575 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:51.985767 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.986353 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.987719 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.988441 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.988518 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.988533 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.989252 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.989305 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:42:51.989136 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.984988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46127 (local address 127.20.105.67:39679)
0808 00:42:51.985056 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.985073 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.985142 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.985388 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.985389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.985392 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:51.985494 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.985496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.986378 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.986380 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.987757 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.987761 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.988605 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.988606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.988693 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.988696 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.988741 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.988778 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":33}
I20260808 00:42:51.989952 32432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672c859e2afb45c8a99d550cebafcb15" candidate_uuid: "264eb8e4d312471993bb5b9f89e9722f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb35618b1ef44514ab63594cc2921b4a" is_pre_election: true
I20260808 00:42:51.990113 32432 raft_consensus.cc:2454] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 264eb8e4d312471993bb5b9f89e9722f in term 0.
I20260808 00:42:51.990357 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672c859e2afb45c8a99d550cebafcb15" candidate_uuid: "264eb8e4d312471993bb5b9f89e9722f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" is_pre_election: true
I20260808 00:42:51.990540 32640 raft_consensus.cc:2454] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 264eb8e4d312471993bb5b9f89e9722f in term 0.
I20260808 00:42:51.990911 32744 leader_election.cc:302] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 264eb8e4d312471993bb5b9f89e9722f, 9d13faf37abd4b0c8a108fee6854e0e4; no voters:
I20260808 00:42:51.991132 507 raft_consensus.cc:2781] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:51.991271 507 raft_consensus.cc:482] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:51.991321 507 raft_consensus.cc:3037] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:51.991767 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.987636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:36113 (local address 127.20.105.66:33409)
0808 00:42:51.987922 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.987925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.988066 (+ 141us) server_negotiation.cc:407] Connection header received
0808 00:42:51.988166 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.988167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.988179 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.988249 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:51.988474 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.989294 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.990727 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.991356 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.991442 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.991452 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.991633 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.991687 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":76,"threads_started":1}
I20260808 00:42:51.991775 494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.987594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:55769 (local address 127.20.105.67:38925)
0808 00:42:51.987637 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:51.987639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:51.988092 (+ 453us) server_negotiation.cc:407] Connection header received
0808 00:42:51.988211 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:51.988212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:51.988223 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:51.988256 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:51.988476 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.989292 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.990801 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.991378 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.991441 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.991452 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:51.991680 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 00:42:51.991718 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:42:51.991797 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.987356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:33409 (local address 127.20.105.65:36113)
0808 00:42:51.987922 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.987935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.988006 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.988300 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.988301 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.988302 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:51.988388 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.988390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.989392 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.989393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.990643 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.990647 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.991435 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.991438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.991520 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.991527 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.991610 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.991663 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":31,"thread_start_us":146,"threads_started":1}
I20260808 00:42:51.992316 509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:51.987356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:38925 (local address 127.20.105.65:55769)
0808 00:42:51.987982 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:51.987993 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:51.988120 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:51.988320 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:51.988321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:51.988322 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:51.988390 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.988391 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.989408 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.989409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.990723 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:51.990727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:51.991489 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:51.991490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:51.991568 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:51.991572 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:42:51.991884 (+ 312us) client_negotiation.cc:240] Negotiation successful
0808 00:42:51.991937 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":3,"thread_start_us":127,"threads_started":1}
I20260808 00:42:51.992563 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672c859e2afb45c8a99d550cebafcb15" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" is_pre_election: true
I20260808 00:42:51.992671 32640 raft_consensus.cc:2454] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 0.
I20260808 00:42:51.992710 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672c859e2afb45c8a99d550cebafcb15" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f" is_pre_election: true
I20260808 00:42:51.992835 32336 leader_election.cc:302] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d13faf37abd4b0c8a108fee6854e0e4, eb35618b1ef44514ab63594cc2921b4a; no voters:
I20260808 00:42:51.993008 504 raft_consensus.cc:2781] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:51.993101 504 raft_consensus.cc:482] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:51.993131 504 raft_consensus.cc:3037] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:51.993234 507 raft_consensus.cc:504] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.993407 382 raft_consensus.cc:2379] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb35618b1ef44514ab63594cc2921b4a in current term 1: Already voted for candidate 264eb8e4d312471993bb5b9f89e9722f in this term.
I20260808 00:42:51.993634 507 leader_election.cc:288] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [CANDIDATE]: Term 1 election: Requested vote from peers eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127), 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409)
I20260808 00:42:51.993834 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672c859e2afb45c8a99d550cebafcb15" candidate_uuid: "264eb8e4d312471993bb5b9f89e9722f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4"
I20260808 00:42:51.993932 32640 raft_consensus.cc:3037] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:51.993980 32432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672c859e2afb45c8a99d550cebafcb15" candidate_uuid: "264eb8e4d312471993bb5b9f89e9722f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb35618b1ef44514ab63594cc2921b4a"
I20260808 00:42:51.994237 504 raft_consensus.cc:504] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.994418 504 leader_election.cc:288] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 election: Requested vote from peers 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925), 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409)
I20260808 00:42:51.994500 32432 raft_consensus.cc:2379] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 264eb8e4d312471993bb5b9f89e9722f in current term 1: Already voted for candidate eb35618b1ef44514ab63594cc2921b4a in this term.
I20260808 00:42:51.994570 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672c859e2afb45c8a99d550cebafcb15" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f"
I20260808 00:42:51.994686 382 raft_consensus.cc:2379] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eb35618b1ef44514ab63594cc2921b4a in current term 1: Already voted for candidate 264eb8e4d312471993bb5b9f89e9722f in this term.
I20260808 00:42:51.995065 32640 raft_consensus.cc:2454] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 264eb8e4d312471993bb5b9f89e9722f in term 1.
I20260808 00:42:51.995282 32744 leader_election.cc:302] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 264eb8e4d312471993bb5b9f89e9722f, 9d13faf37abd4b0c8a108fee6854e0e4; no voters: eb35618b1ef44514ab63594cc2921b4a
I20260808 00:42:51.995307 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "672c859e2afb45c8a99d550cebafcb15" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4"
I20260808 00:42:51.995422 32640 raft_consensus.cc:2379] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate eb35618b1ef44514ab63594cc2921b4a in current term 1: Already voted for candidate 264eb8e4d312471993bb5b9f89e9722f in this term.
I20260808 00:42:51.995436 507 raft_consensus.cc:2781] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:51.995657 507 raft_consensus.cc:686] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 1 LEADER]: Becoming Leader. State: Replica: 264eb8e4d312471993bb5b9f89e9722f, State: Running, Role: LEADER
I20260808 00:42:51.995663 32336 leader_election.cc:302] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb35618b1ef44514ab63594cc2921b4a; no voters: 264eb8e4d312471993bb5b9f89e9722f, 9d13faf37abd4b0c8a108fee6854e0e4
I20260808 00:42:51.995767 504 raft_consensus.cc:2726] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 00:42:51.995800 507 consensus_queue.cc:237] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } }
I20260808 00:42:51.997044 32206 catalog_manager.cc:5495] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f reported cstate change: term changed from 0 to 1, leader changed from <none> to 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67). New cstate: current_term: 1 leader_uuid: "264eb8e4d312471993bb5b9f89e9722f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:52.014995 494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.011861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37518 (local address 127.20.105.67:38925)
0808 00:42:52.011935 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.011937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.011948 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:52.012002 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.012003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.012072 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.012159 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:52.012381 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.013011 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.013946 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.014188 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.014298 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.014532 (+ 234us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:52.014730 (+ 198us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:52.014835 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.014854 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.014881 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:42:52.017887 32432 raft_consensus.cc:1261] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Refusing update from remote peer 264eb8e4d312471993bb5b9f89e9722f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.017876 32640 raft_consensus.cc:1261] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Refusing update from remote peer 264eb8e4d312471993bb5b9f89e9722f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.018254 507 consensus_queue.cc:1035] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.018479 515 consensus_queue.cc:1035] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.023002 520 mvcc.cc:205] Tried to move back new op lower bound from 7316069466181226496 to 7316069466095845376. Current Snapshot: MvccSnapshot[applied={T|T < 7316069466181226496 or (T in {7316069466181226496})}]
I20260808 00:42:52.025178 32205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:52.025345 32205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:52.026497 32205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: Sending DeleteTablet for 3 replicas of tablet 672c859e2afb45c8a99d550cebafcb15
I20260808 00:42:52.026895 337 tablet_service.cc:1514] Processing DeleteTablet for tablet 672c859e2afb45c8a99d550cebafcb15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37508
I20260808 00:42:52.026887 32406 tablet_service.cc:1514] Processing DeleteTablet for tablet 672c859e2afb45c8a99d550cebafcb15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:60528
I20260808 00:42:52.027200 536 tablet_replica.cc:330] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: stopping tablet replica
I20260808 00:42:52.027184 32205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 00:42:52.027334 536 raft_consensus.cc:2229] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260808 00:42:52.027375 32205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:52.027452 536 raft_consensus.cc:2258] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.027782 537 tablet_replica.cc:330] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: stopping tablet replica
I20260808 00:42:52.027904 537 raft_consensus.cc:2229] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:52.028082 32610 tablet_service.cc:1514] Processing DeleteTablet for tablet 672c859e2afb45c8a99d550cebafcb15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37432
I20260808 00:42:52.028364 539 tablet_replica.cc:330] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: stopping tablet replica
I20260808 00:42:52.028680 536 ts_tablet_manager.cc:1905] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.029486 32406 tablet_service.cc:1467] Processing CreateTablet for tablet eefdda9f04ce437ab5d71e1bdc6a3530 (DEFAULT_TABLE table=test-table [id=756c4e102ba4475da881a670078148e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.029523 337 tablet_service.cc:1467] Processing CreateTablet for tablet eefdda9f04ce437ab5d71e1bdc6a3530 (DEFAULT_TABLE table=test-table [id=756c4e102ba4475da881a670078148e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.029644 337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eefdda9f04ce437ab5d71e1bdc6a3530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.029644 32406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eefdda9f04ce437ab5d71e1bdc6a3530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.031360 500 tablet_bootstrap.cc:492] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: Bootstrap starting.
I20260808 00:42:52.031543 32610 tablet_service.cc:1467] Processing CreateTablet for tablet eefdda9f04ce437ab5d71e1bdc6a3530 (DEFAULT_TABLE table=test-table [id=756c4e102ba4475da881a670078148e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.031688 32610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eefdda9f04ce437ab5d71e1bdc6a3530. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.031754 497 tablet_bootstrap.cc:492] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: Bootstrap starting.
I20260808 00:42:52.031929 500 tablet_bootstrap.cc:654] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.032363 497 tablet_bootstrap.cc:654] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.033179 497 tablet_bootstrap.cc:492] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: No bootstrap required, opened a new log
I20260808 00:42:52.033236 497 ts_tablet_manager.cc:1397] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:52.033427 497 raft_consensus.cc:348] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.033540 497 raft_consensus.cc:374] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.033568 497 raft_consensus.cc:729] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb35618b1ef44514ab63594cc2921b4a, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.033587 498 tablet_bootstrap.cc:492] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: Bootstrap starting.
I20260808 00:42:52.033623 497 consensus_queue.cc:260] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.033726 497 ts_tablet_manager.cc:1428] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:42:52.034107 498 tablet_bootstrap.cc:654] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.034586 539 raft_consensus.cc:2229] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:52.034771 539 raft_consensus.cc:2258] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.035392 539 ts_tablet_manager.cc:1905] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.035425 536 ts_tablet_manager.cc:1918] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:52.035508 536 log.cc:1196] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/672c859e2afb45c8a99d550cebafcb15
I20260808 00:42:52.035825 536 ts_tablet_manager.cc:1939] T 672c859e2afb45c8a99d550cebafcb15 P eb35618b1ef44514ab63594cc2921b4a: Deleting consensus metadata
I20260808 00:42:52.036182 537 raft_consensus.cc:2258] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.036371 32186 catalog_manager.cc:4841] TS eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127): tablet 672c859e2afb45c8a99d550cebafcb15 (table test-table [id=44e8ff6949d249acb14d9354174a0507]) successfully deleted
I20260808 00:42:52.036785 498 tablet_bootstrap.cc:492] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: No bootstrap required, opened a new log
I20260808 00:42:52.036849 498 ts_tablet_manager.cc:1397] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 00:42:52.036885 537 ts_tablet_manager.cc:1905] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.037154 498 raft_consensus.cc:348] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.037272 498 raft_consensus.cc:374] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.037302 498 raft_consensus.cc:729] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d13faf37abd4b0c8a108fee6854e0e4, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.037358 498 consensus_queue.cc:260] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.037472 498 ts_tablet_manager.cc:1428] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:52.038709 539 ts_tablet_manager.cc:1918] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:52.038821 539 log.cc:1196] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/672c859e2afb45c8a99d550cebafcb15
I20260808 00:42:52.039091 537 ts_tablet_manager.cc:1918] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:52.039173 537 log.cc:1196] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/672c859e2afb45c8a99d550cebafcb15
I20260808 00:42:52.039203 539 ts_tablet_manager.cc:1939] T 672c859e2afb45c8a99d550cebafcb15 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting consensus metadata
I20260808 00:42:52.039438 537 ts_tablet_manager.cc:1939] T 672c859e2afb45c8a99d550cebafcb15 P 264eb8e4d312471993bb5b9f89e9722f: Deleting consensus metadata
I20260808 00:42:52.039752 32186 catalog_manager.cc:4841] TS 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409): tablet 672c859e2afb45c8a99d550cebafcb15 (table test-table [id=44e8ff6949d249acb14d9354174a0507]) successfully deleted
I20260808 00:42:52.039881 32189 catalog_manager.cc:4841] TS 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925): tablet 672c859e2afb45c8a99d550cebafcb15 (table test-table [id=44e8ff6949d249acb14d9354174a0507]) successfully deleted
I20260808 00:42:52.040521 500 tablet_bootstrap.cc:492] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: No bootstrap required, opened a new log
I20260808 00:42:52.040597 500 ts_tablet_manager.cc:1397] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260808 00:42:52.040781 500 raft_consensus.cc:348] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.040868 500 raft_consensus.cc:374] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.040891 500 raft_consensus.cc:729] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264eb8e4d312471993bb5b9f89e9722f, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.040940 500 consensus_queue.cc:260] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.041034 500 ts_tablet_manager.cc:1428] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:52.116819 504 raft_consensus.cc:482] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:52.117018 504 raft_consensus.cc:504] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.117312 504 leader_election.cc:288] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409), 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:52.117599 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eefdda9f04ce437ab5d71e1bdc6a3530" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f" is_pre_election: true
I20260808 00:42:52.117590 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eefdda9f04ce437ab5d71e1bdc6a3530" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" is_pre_election: true
I20260808 00:42:52.117724 32640 raft_consensus.cc:2454] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 0.
I20260808 00:42:52.117726 382 raft_consensus.cc:2454] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 0.
I20260808 00:42:52.117910 32336 leader_election.cc:302] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 264eb8e4d312471993bb5b9f89e9722f, 9d13faf37abd4b0c8a108fee6854e0e4, eb35618b1ef44514ab63594cc2921b4a; no voters:
I20260808 00:42:52.118099 504 raft_consensus.cc:2781] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:52.118144 504 raft_consensus.cc:482] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:52.118160 504 raft_consensus.cc:3037] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.119100 504 raft_consensus.cc:504] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.119272 504 leader_election.cc:288] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 election: Requested vote from peers 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409), 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:52.119488 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eefdda9f04ce437ab5d71e1bdc6a3530" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f"
I20260808 00:42:52.119478 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eefdda9f04ce437ab5d71e1bdc6a3530" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4"
I20260808 00:42:52.119577 382 raft_consensus.cc:3037] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.119577 32640 raft_consensus.cc:3037] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.120589 382 raft_consensus.cc:2454] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 1.
I20260808 00:42:52.120589 32640 raft_consensus.cc:2454] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 1.
I20260808 00:42:52.120805 32336 leader_election.cc:302] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d13faf37abd4b0c8a108fee6854e0e4, eb35618b1ef44514ab63594cc2921b4a; no voters:
I20260808 00:42:52.120910 504 raft_consensus.cc:2781] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:52.121064 504 raft_consensus.cc:686] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 1 LEADER]: Becoming Leader. State: Replica: eb35618b1ef44514ab63594cc2921b4a, State: Running, Role: LEADER
I20260808 00:42:52.121150 504 consensus_queue.cc:237] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.122298 32205 catalog_manager.cc:5495] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a reported cstate change: term changed from 0 to 1, leader changed from <none> to eb35618b1ef44514ab63594cc2921b4a (127.20.105.65). New cstate: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } health_report { overall_health: UNKNOWN } } }
Aug 08 00:42:52 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149770, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.126@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:52.162850 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.153241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34936 (local address 127.20.105.126:44659)
0808 00:42:52.153411 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.153414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.153541 (+ 127us) server_negotiation.cc:407] Connection header received
0808 00:42:52.153622 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.153622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.153635 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.153716 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:52.154597 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.155188 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.156231 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.156460 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.156540 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.159654 (+ 3114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:52.159655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:52.159656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:52.159696 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:52.161420 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:52.161900 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:52.161901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:52.161902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:52.161975 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:52.162189 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:52.162190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:52.162191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:52.162338 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:52.162424 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.162595 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.162704 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260808 00:42:52.162904 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.153148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44659 (local address 127.0.0.1:34936)
0808 00:42:52.153417 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:52.153432 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:52.153547 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:52.153756 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:52.153757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:52.154387 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:52.154491 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.154495 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.155300 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.155301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.156105 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.156108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.156559 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.156560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.156618 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.157405 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:52.157412 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:52.159501 (+ 2089us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:52.161548 (+ 2047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:52.161560 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:52.161563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:52.161805 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:52.162050 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:52.162050 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:52.162051 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:52.162123 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:52.162411 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:52.162421 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:52.162476 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 00:42:52.162566 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 00:42:52.162782 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":88,"thread_start_us":143,"threads_started":1}
I20260808 00:42:52.165563 32205 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:34936:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:42:52.165890 32205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:52.167951 32610 tablet_service.cc:1467] Processing CreateTablet for tablet 7a75be49158145998f997e825313ea1b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=648e8495f2654a429ba389e98dfc2e3a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:52.167968 32406 tablet_service.cc:1467] Processing CreateTablet for tablet 7a75be49158145998f997e825313ea1b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=648e8495f2654a429ba389e98dfc2e3a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:52.168080 337 tablet_service.cc:1467] Processing CreateTablet for tablet 7a75be49158145998f997e825313ea1b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=648e8495f2654a429ba389e98dfc2e3a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:42:52.168233 32406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a75be49158145998f997e825313ea1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.168232 32610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a75be49158145998f997e825313ea1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.168244 337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a75be49158145998f997e825313ea1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.170073 498 tablet_bootstrap.cc:492] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: Bootstrap starting.
I20260808 00:42:52.170395 497 tablet_bootstrap.cc:492] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: Bootstrap starting.
I20260808 00:42:52.170380 500 tablet_bootstrap.cc:492] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: Bootstrap starting.
I20260808 00:42:52.170737 498 tablet_bootstrap.cc:654] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.170974 497 tablet_bootstrap.cc:654] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.171142 500 tablet_bootstrap.cc:654] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.171798 498 tablet_bootstrap.cc:492] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: No bootstrap required, opened a new log
I20260808 00:42:52.171868 498 ts_tablet_manager.cc:1397] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:42:52.171871 497 tablet_bootstrap.cc:492] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: No bootstrap required, opened a new log
I20260808 00:42:52.171931 497 ts_tablet_manager.cc:1397] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:42:52.172096 498 raft_consensus.cc:348] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.172214 498 raft_consensus.cc:374] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.172181 497 raft_consensus.cc:348] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.172246 498 raft_consensus.cc:729] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d13faf37abd4b0c8a108fee6854e0e4, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.172263 497 raft_consensus.cc:374] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.172279 497 raft_consensus.cc:729] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb35618b1ef44514ab63594cc2921b4a, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.172297 498 consensus_queue.cc:260] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.172364 497 consensus_queue.cc:260] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.172415 498 ts_tablet_manager.cc:1428] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:52.172487 497 ts_tablet_manager.cc:1428] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:52.172607 505 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.172595 544 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.172710 544 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:52.172709 505 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:52.173161 500 tablet_bootstrap.cc:492] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: No bootstrap required, opened a new log
I20260808 00:42:52.173259 500 ts_tablet_manager.cc:1397] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 00:42:52.173413 500 raft_consensus.cc:348] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.173489 500 raft_consensus.cc:374] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.173503 500 raft_consensus.cc:729] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264eb8e4d312471993bb5b9f89e9722f, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.173584 500 consensus_queue.cc:260] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.173715 500 ts_tablet_manager.cc:1428] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:52.173761 515 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.173877 515 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:52.222752 544 raft_consensus.cc:482] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:52.222949 544 raft_consensus.cc:504] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.223198 544 leader_election.cc:288] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409), 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:52.223544 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a75be49158145998f997e825313ea1b" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f" is_pre_election: true
I20260808 00:42:52.223583 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a75be49158145998f997e825313ea1b" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" is_pre_election: true
I20260808 00:42:52.223716 382 raft_consensus.cc:2454] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 0.
I20260808 00:42:52.223709 32640 raft_consensus.cc:2454] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 0.
I20260808 00:42:52.224058 32339 leader_election.cc:302] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 264eb8e4d312471993bb5b9f89e9722f, eb35618b1ef44514ab63594cc2921b4a; no voters:
I20260808 00:42:52.224215 544 raft_consensus.cc:2781] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:52.224269 544 raft_consensus.cc:482] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:52.224289 544 raft_consensus.cc:3037] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.225296 544 raft_consensus.cc:504] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.225642 544 leader_election.cc:288] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 election: Requested vote from peers 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409), 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:52.225849 382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a75be49158145998f997e825313ea1b" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f"
I20260808 00:42:52.225865 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a75be49158145998f997e825313ea1b" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4"
I20260808 00:42:52.225994 32640 raft_consensus.cc:3037] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.226003 382 raft_consensus.cc:3037] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.226791 32640 raft_consensus.cc:2454] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 1.
I20260808 00:42:52.226826 382 raft_consensus.cc:2454] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 1.
I20260808 00:42:52.227057 32336 leader_election.cc:302] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d13faf37abd4b0c8a108fee6854e0e4, eb35618b1ef44514ab63594cc2921b4a; no voters:
I20260808 00:42:52.227205 544 raft_consensus.cc:2781] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:52.227310 544 raft_consensus.cc:686] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [term 1 LEADER]: Becoming Leader. State: Replica: eb35618b1ef44514ab63594cc2921b4a, State: Running, Role: LEADER
I20260808 00:42:52.227384 544 consensus_queue.cc:237] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.227838 504 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: TxnStatusTablet state changed. Reason: New leader eb35618b1ef44514ab63594cc2921b4a. Latest consensus state: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.227968 504 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:52.228323 563 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:42:52.228495 563 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 00:42:52.228389 32206 catalog_manager.cc:5495] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a reported cstate change: term changed from 0 to 1, leader changed from <none> to eb35618b1ef44514ab63594cc2921b4a (127.20.105.65). New cstate: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:52.238461 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.234826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46127 (local address 127.0.0.1:60534)
0808 00:42:52.234933 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:52.234947 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:52.235001 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:52.235426 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:52.235428 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:52.235429 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:52.235518 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.235520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.236666 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.236667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.237323 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.237325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.237856 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.237856 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.237904 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.237923 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:52.238286 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:52.238294 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 00:42:52.238326 (+ 32us) client_negotiation.cc:240] Negotiation successful
0808 00:42:52.238386 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 00:42:52.238461 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.235117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60534 (local address 127.20.105.65:46127)
0808 00:42:52.235194 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.235196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.235207 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:52.235252 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.235253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.235281 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.235378 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:52.235614 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.236573 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.237420 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.237770 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.237857 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.238034 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:52.238210 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:52.238304 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.238332 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.238359 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:42:52.240126 32640 raft_consensus.cc:1261] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Refusing update from remote peer eb35618b1ef44514ab63594cc2921b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.240187 382 raft_consensus.cc:1261] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Refusing update from remote peer eb35618b1ef44514ab63594cc2921b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.240477 544 consensus_queue.cc:1035] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.240658 504 consensus_queue.cc:1035] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.241407 505 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: TxnStatusTablet state changed. Reason: New leader eb35618b1ef44514ab63594cc2921b4a. Latest consensus state: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.241523 505 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:52.241497 515 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: TxnStatusTablet state changed. Reason: New leader eb35618b1ef44514ab63594cc2921b4a. Latest consensus state: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.241623 515 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:52.242662 504 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.242756 544 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.242842 544 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:52.242911 521 mvcc.cc:205] Tried to move back new op lower bound from 7316069467092418560 to 7316069467044016128. Current Snapshot: MvccSnapshot[applied={T|T < 7316069467092418560}]
I20260808 00:42:52.242766 504 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P eb35618b1ef44514ab63594cc2921b4a: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:42:52.243680 505 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.243784 505 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P 9d13faf37abd4b0c8a108fee6854e0e4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:52.244431 518 mvcc.cc:205] Tried to move back new op lower bound from 7316069467092418560 to 7316069467044016128. Current Snapshot: MvccSnapshot[applied={T|T < 7316069467092418560}]
I20260808 00:42:52.244544 515 tablet_replica.cc:439] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } } }
I20260808 00:42:52.244638 515 tablet_replica.cc:442] T 7a75be49158145998f997e825313ea1b P 264eb8e4d312471993bb5b9f89e9722f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:42:52.248625 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.245543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.20.105.65:46127)
0808 00:42:52.245589 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.245591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.245626 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:42:52.245703 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.245705 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.245717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.245774 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:52.245939 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.246506 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.247679 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.248006 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.248095 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.248308 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:52.248353 (+ 45us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:52.248423 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.248498 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.248526 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:42:52.255440 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.252223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:40541 (local address 127.20.105.65:46127)
0808 00:42:52.252291 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.252294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.252501 (+ 207us) server_negotiation.cc:407] Connection header received
0808 00:42:52.252575 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.252576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.252586 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.252634 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:52.252826 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.253321 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.254655 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.254839 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.254893 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.255151 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:52.255183 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:52.255218 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.255318 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.255338 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:42:52.255587 574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.252167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46127 (local address 127.20.105.65:40541)
0808 00:42:52.252413 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:52.252428 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:52.252509 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:52.252666 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:52.252668 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:52.252669 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:42:52.252762 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.252764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.253410 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.253412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.254556 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.254559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.254934 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.254936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.255026 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.255051 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:42:52.255251 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:42:52.255253 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:42:52.255311 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:42:52.255367 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":127,"threads_started":1}
I20260808 00:42:52.258599 32640 raft_consensus.cc:1261] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Refusing update from remote peer eb35618b1ef44514ab63594cc2921b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.258605 382 raft_consensus.cc:1261] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Refusing update from remote peer eb35618b1ef44514ab63594cc2921b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.258870 569 consensus_queue.cc:1035] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.258991 544 consensus_queue.cc:1035] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.273479 32206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:52.273658 32206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:52.274530 32206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: Sending DeleteTablet for 3 replicas of tablet eefdda9f04ce437ab5d71e1bdc6a3530
I20260808 00:42:52.274801 32406 tablet_service.cc:1514] Processing DeleteTablet for tablet eefdda9f04ce437ab5d71e1bdc6a3530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:60528
I20260808 00:42:52.274856 32610 tablet_service.cc:1514] Processing DeleteTablet for tablet eefdda9f04ce437ab5d71e1bdc6a3530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37432
I20260808 00:42:52.274933 536 tablet_replica.cc:330] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: stopping tablet replica
I20260808 00:42:52.275007 536 raft_consensus.cc:2229] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:52.274998 539 tablet_replica.cc:330] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: stopping tablet replica
I20260808 00:42:52.275085 539 raft_consensus.cc:2229] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:52.275102 536 raft_consensus.cc:2258] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.275128 539 raft_consensus.cc:2258] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.275204 337 tablet_service.cc:1514] Processing DeleteTablet for tablet eefdda9f04ce437ab5d71e1bdc6a3530 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37508
I20260808 00:42:52.275307 537 tablet_replica.cc:330] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: stopping tablet replica
I20260808 00:42:52.275372 537 raft_consensus.cc:2229] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:52.275417 537 raft_consensus.cc:2258] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.275585 539 ts_tablet_manager.cc:1905] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.275588 536 ts_tablet_manager.cc:1905] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.275732 537 ts_tablet_manager.cc:1905] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.277837 539 ts_tablet_manager.cc:1918] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:52.277885 536 ts_tablet_manager.cc:1918] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:52.277905 539 log.cc:1196] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/eefdda9f04ce437ab5d71e1bdc6a3530
I20260808 00:42:52.277942 536 log.cc:1196] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/eefdda9f04ce437ab5d71e1bdc6a3530
I20260808 00:42:52.278170 539 ts_tablet_manager.cc:1939] T eefdda9f04ce437ab5d71e1bdc6a3530 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting consensus metadata
I20260808 00:42:52.278215 536 ts_tablet_manager.cc:1939] T eefdda9f04ce437ab5d71e1bdc6a3530 P eb35618b1ef44514ab63594cc2921b4a: Deleting consensus metadata
I20260808 00:42:52.278235 537 ts_tablet_manager.cc:1918] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:52.278317 537 log.cc:1196] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/eefdda9f04ce437ab5d71e1bdc6a3530
I20260808 00:42:52.278597 537 ts_tablet_manager.cc:1939] T eefdda9f04ce437ab5d71e1bdc6a3530 P 264eb8e4d312471993bb5b9f89e9722f: Deleting consensus metadata
I20260808 00:42:52.278738 32186 catalog_manager.cc:4841] TS eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127): tablet eefdda9f04ce437ab5d71e1bdc6a3530 (table test-table [id=756c4e102ba4475da881a670078148e5]) successfully deleted
I20260808 00:42:52.279327 32189 catalog_manager.cc:4841] TS 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925): tablet eefdda9f04ce437ab5d71e1bdc6a3530 (table test-table [id=756c4e102ba4475da881a670078148e5]) successfully deleted
I20260808 00:42:52.279619 32186 catalog_manager.cc:4841] TS 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409): tablet eefdda9f04ce437ab5d71e1bdc6a3530 (table test-table [id=756c4e102ba4475da881a670078148e5]) successfully deleted
Aug 08 00:42:52 dist-test-slave-kvfs krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:42:52 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:52 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:42:52.307641 558 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:52.300622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.20.105.126:44659)
0808 00:42:52.300717 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.300721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.300820 (+ 99us) server_negotiation.cc:407] Connection header received
0808 00:42:52.300913 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.300915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.300934 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.301003 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:52.301700 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.302593 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.303387 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.303733 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.303812 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.307532 (+ 3720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34946: BlockingRecv error: recv got EOF from 127.0.0.1:34946 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 00:42:52 dist-test-slave-kvfs krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM
I20260808 00:42:52.318342 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.308924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.20.105.126:44659)
0808 00:42:52.308972 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.308974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.309037 (+ 63us) server_negotiation.cc:407] Connection header received
0808 00:42:52.309129 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.309130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.309139 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.309189 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:52.309880 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.310425 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.311504 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.311917 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.312076 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.315238 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:52.315241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:52.315241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:52.315251 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:52.317011 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:52.317415 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:52.317417 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:52.317418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:52.317486 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:52.317706 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:52.317707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:52.317707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:52.317852 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:52.317997 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.318102 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.318206 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:42:52.320379 32206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:42:52.320662 32206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:52.322345 32406 tablet_service.cc:1467] Processing CreateTablet for tablet 121a80e541634a6f9b178486fc77e51f (DEFAULT_TABLE table=test-table [id=96cb719b0c91415d9c5bf8ab2d4ea1c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.322408 32610 tablet_service.cc:1467] Processing CreateTablet for tablet 121a80e541634a6f9b178486fc77e51f (DEFAULT_TABLE table=test-table [id=96cb719b0c91415d9c5bf8ab2d4ea1c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.322547 32406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121a80e541634a6f9b178486fc77e51f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.322597 32610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121a80e541634a6f9b178486fc77e51f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.323972 498 tablet_bootstrap.cc:492] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: Bootstrap starting.
I20260808 00:42:52.324110 337 tablet_service.cc:1467] Processing CreateTablet for tablet 121a80e541634a6f9b178486fc77e51f (DEFAULT_TABLE table=test-table [id=96cb719b0c91415d9c5bf8ab2d4ea1c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.324272 337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 121a80e541634a6f9b178486fc77e51f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.324597 498 tablet_bootstrap.cc:654] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.325443 498 tablet_bootstrap.cc:492] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: No bootstrap required, opened a new log
I20260808 00:42:52.325502 498 ts_tablet_manager.cc:1397] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:52.325707 498 raft_consensus.cc:348] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.325798 498 raft_consensus.cc:374] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.325820 498 raft_consensus.cc:729] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d13faf37abd4b0c8a108fee6854e0e4, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.325881 498 consensus_queue.cc:260] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.325997 498 ts_tablet_manager.cc:1428] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:52.326028 500 tablet_bootstrap.cc:492] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: Bootstrap starting.
I20260808 00:42:52.326087 497 tablet_bootstrap.cc:492] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: Bootstrap starting.
I20260808 00:42:52.326469 497 tablet_bootstrap.cc:654] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.326501 500 tablet_bootstrap.cc:654] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.327512 500 tablet_bootstrap.cc:492] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: No bootstrap required, opened a new log
I20260808 00:42:52.327591 500 ts_tablet_manager.cc:1397] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:52.327638 497 tablet_bootstrap.cc:492] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: No bootstrap required, opened a new log
I20260808 00:42:52.327698 497 ts_tablet_manager.cc:1397] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:52.327744 500 raft_consensus.cc:348] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.327829 500 raft_consensus.cc:374] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.327850 500 raft_consensus.cc:729] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264eb8e4d312471993bb5b9f89e9722f, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.327898 500 consensus_queue.cc:260] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.327873 497 raft_consensus.cc:348] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.327951 497 raft_consensus.cc:374] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.327980 497 raft_consensus.cc:729] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb35618b1ef44514ab63594cc2921b4a, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.328050 500 ts_tablet_manager.cc:1428] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:52.328055 497 consensus_queue.cc:260] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.328164 497 ts_tablet_manager.cc:1428] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:52.384542 505 raft_consensus.cc:482] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:52.384708 505 raft_consensus.cc:504] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.385231 505 leader_election.cc:288] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127), 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:52.389705 603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.385487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:38925 (local address 127.20.105.66:59071)
0808 00:42:52.385935 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:52.385952 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:52.386032 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:52.386195 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:52.386196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:52.386198 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:52.386317 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.386320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.387381 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.387382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.388514 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.388518 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.389367 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.389368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.389455 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.389462 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:42:52.389530 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 00:42:52.389580 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":211,"thread_start_us":159,"threads_started":1}
I20260808 00:42:52.389698 494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.385613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:59071 (local address 127.20.105.67:38925)
0808 00:42:52.385682 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.385685 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.386050 (+ 365us) server_negotiation.cc:407] Connection header received
0808 00:42:52.386094 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.386095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.386108 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.386175 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:52.386452 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.387236 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.388623 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.389255 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.389347 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.389360 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.389541 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.389609 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:42:52.389783 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.385593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:35635 (local address 127.20.105.65:46127)
0808 00:42:52.385662 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.385665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.385851 (+ 186us) server_negotiation.cc:407] Connection header received
0808 00:42:52.385952 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.385954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.385973 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.386046 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:42:52.386712 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.387274 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.388719 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.389368 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.389440 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.389451 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.389645 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.389692 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:42:52.390084 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.385486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46127 (local address 127.20.105.66:35635)
0808 00:42:52.385738 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:52.385752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:52.385847 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:52.386116 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:52.386117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:52.386119 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:42:52.386599 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.386602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.387399 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.387400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.388631 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:52.388635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.389454 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.389455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:52.389548 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.389553 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:42:52.389970 (+ 417us) client_negotiation.cc:240] Negotiation successful
0808 00:42:52.390016 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":44,"thread_start_us":134,"threads_started":1}
I20260808 00:42:52.390679 384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121a80e541634a6f9b178486fc77e51f" candidate_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f" is_pre_election: true
I20260808 00:42:52.390698 32432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121a80e541634a6f9b178486fc77e51f" candidate_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb35618b1ef44514ab63594cc2921b4a" is_pre_election: true
I20260808 00:42:52.390810 384 raft_consensus.cc:2454] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d13faf37abd4b0c8a108fee6854e0e4 in term 0.
I20260808 00:42:52.390810 32432 raft_consensus.cc:2454] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d13faf37abd4b0c8a108fee6854e0e4 in term 0.
I20260808 00:42:52.391072 32534 leader_election.cc:302] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d13faf37abd4b0c8a108fee6854e0e4, eb35618b1ef44514ab63594cc2921b4a; no voters:
I20260808 00:42:52.391258 505 raft_consensus.cc:2781] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:52.391305 505 raft_consensus.cc:482] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:52.391336 505 raft_consensus.cc:3037] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.392318 505 raft_consensus.cc:504] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.392537 505 leader_election.cc:288] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [CANDIDATE]: Term 1 election: Requested vote from peers eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127), 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:52.392714 384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121a80e541634a6f9b178486fc77e51f" candidate_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f"
I20260808 00:42:52.392814 384 raft_consensus.cc:3037] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.393661 384 raft_consensus.cc:2454] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d13faf37abd4b0c8a108fee6854e0e4 in term 1.
I20260808 00:42:52.393810 32432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "121a80e541634a6f9b178486fc77e51f" candidate_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb35618b1ef44514ab63594cc2921b4a"
I20260808 00:42:52.393867 32538 leader_election.cc:302] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 264eb8e4d312471993bb5b9f89e9722f, 9d13faf37abd4b0c8a108fee6854e0e4; no voters:
I20260808 00:42:52.393909 32432 raft_consensus.cc:3037] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.393983 505 raft_consensus.cc:2781] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:52.394196 505 raft_consensus.cc:686] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 LEADER]: Becoming Leader. State: Replica: 9d13faf37abd4b0c8a108fee6854e0e4, State: Running, Role: LEADER
I20260808 00:42:52.394285 505 consensus_queue.cc:237] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.394932 32432 raft_consensus.cc:2454] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d13faf37abd4b0c8a108fee6854e0e4 in term 1.
I20260808 00:42:52.395565 32206 catalog_manager.cc:5495] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66). New cstate: current_term: 1 leader_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:52.405776 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.402139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37438 (local address 127.20.105.66:33409)
0808 00:42:52.402199 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.402201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.402211 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:42:52.402293 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.402294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.402316 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.402390 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:52.402624 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.403612 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.404763 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.405019 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.405109 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.405329 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:52.405485 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:52.405592 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.405625 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.405655 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:42:52.407114 384 raft_consensus.cc:1261] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Refusing update from remote peer 9d13faf37abd4b0c8a108fee6854e0e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.407124 32432 raft_consensus.cc:1261] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Refusing update from remote peer 9d13faf37abd4b0c8a108fee6854e0e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.407476 604 consensus_queue.cc:1035] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.407698 505 consensus_queue.cc:1035] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.412883 32206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:52.413026 32206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:52.413901 32206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: Sending DeleteTablet for 3 replicas of tablet 121a80e541634a6f9b178486fc77e51f
I20260808 00:42:52.414208 32406 tablet_service.cc:1514] Processing DeleteTablet for tablet 121a80e541634a6f9b178486fc77e51f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:60528
I20260808 00:42:52.414238 32610 tablet_service.cc:1514] Processing DeleteTablet for tablet 121a80e541634a6f9b178486fc77e51f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37432
I20260808 00:42:52.414335 536 tablet_replica.cc:330] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: stopping tablet replica
I20260808 00:42:52.414345 539 tablet_replica.cc:330] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: stopping tablet replica
I20260808 00:42:52.414391 536 raft_consensus.cc:2229] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:52.414403 539 raft_consensus.cc:2229] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:42:52.414441 536 raft_consensus.cc:2258] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.414520 539 raft_consensus.cc:2258] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.414968 337 tablet_service.cc:1514] Processing DeleteTablet for tablet 121a80e541634a6f9b178486fc77e51f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37508
I20260808 00:42:52.415088 537 tablet_replica.cc:330] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: stopping tablet replica
I20260808 00:42:52.415110 536 ts_tablet_manager.cc:1905] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.415156 537 raft_consensus.cc:2229] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:52.415207 537 raft_consensus.cc:2258] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.415746 32206 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 00:42:52.415941 539 ts_tablet_manager.cc:1905] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 00:42:52.415984 32206 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:42:52.417255 536 ts_tablet_manager.cc:1918] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:52.417337 536 log.cc:1196] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/121a80e541634a6f9b178486fc77e51f
I20260808 00:42:52.417595 536 ts_tablet_manager.cc:1939] T 121a80e541634a6f9b178486fc77e51f P eb35618b1ef44514ab63594cc2921b4a: Deleting consensus metadata
I20260808 00:42:52.417909 32610 tablet_service.cc:1467] Processing CreateTablet for tablet 41cd527462504e8b9e618fea705a4b99 (DEFAULT_TABLE table=test-table [id=ea9de6671ff6447cb9a4aa5cd33f6bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.418031 539 ts_tablet_manager.cc:1918] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:52.418051 32610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41cd527462504e8b9e618fea705a4b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.418088 539 log.cc:1196] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/121a80e541634a6f9b178486fc77e51f
I20260808 00:42:52.418336 539 ts_tablet_manager.cc:1939] T 121a80e541634a6f9b178486fc77e51f P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting consensus metadata
I20260808 00:42:52.418807 32186 catalog_manager.cc:4841] TS 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409): tablet 121a80e541634a6f9b178486fc77e51f (table test-table [id=96cb719b0c91415d9c5bf8ab2d4ea1c7]) successfully deleted
I20260808 00:42:52.419610 498 tablet_bootstrap.cc:492] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: Bootstrap starting.
I20260808 00:42:52.419658 32406 tablet_service.cc:1467] Processing CreateTablet for tablet 41cd527462504e8b9e618fea705a4b99 (DEFAULT_TABLE table=test-table [id=ea9de6671ff6447cb9a4aa5cd33f6bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.419788 32406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41cd527462504e8b9e618fea705a4b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.420122 498 tablet_bootstrap.cc:654] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.420935 498 tablet_bootstrap.cc:492] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: No bootstrap required, opened a new log
I20260808 00:42:52.421001 498 ts_tablet_manager.cc:1397] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:52.421166 498 raft_consensus.cc:348] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.421243 498 raft_consensus.cc:374] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.421264 498 raft_consensus.cc:729] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d13faf37abd4b0c8a108fee6854e0e4, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.421312 498 consensus_queue.cc:260] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.421396 497 tablet_bootstrap.cc:492] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a: Bootstrap starting.
I20260808 00:42:52.421427 498 ts_tablet_manager.cc:1428] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:52.421790 497 tablet_bootstrap.cc:654] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.422365 32186 catalog_manager.cc:4841] TS eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127): tablet 121a80e541634a6f9b178486fc77e51f (table test-table [id=96cb719b0c91415d9c5bf8ab2d4ea1c7]) successfully deleted
I20260808 00:42:52.423187 337 tablet_service.cc:1467] Processing CreateTablet for tablet 41cd527462504e8b9e618fea705a4b99 (DEFAULT_TABLE table=test-table [id=ea9de6671ff6447cb9a4aa5cd33f6bec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:42:52.423349 337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41cd527462504e8b9e618fea705a4b99. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:52.423543 497 tablet_bootstrap.cc:492] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a: No bootstrap required, opened a new log
I20260808 00:42:52.423597 497 ts_tablet_manager.cc:1397] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:42:52.423739 497 raft_consensus.cc:348] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.423830 497 raft_consensus.cc:374] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.423856 497 raft_consensus.cc:729] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb35618b1ef44514ab63594cc2921b4a, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.423914 497 consensus_queue.cc:260] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.424052 497 ts_tablet_manager.cc:1428] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:42:52.425033 500 tablet_bootstrap.cc:492] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: Bootstrap starting.
I20260808 00:42:52.425388 500 tablet_bootstrap.cc:654] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:52.425498 537 ts_tablet_manager.cc:1905] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.427232 537 ts_tablet_manager.cc:1918] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:42:52.427313 537 log.cc:1196] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/121a80e541634a6f9b178486fc77e51f
I20260808 00:42:52.427600 537 ts_tablet_manager.cc:1939] T 121a80e541634a6f9b178486fc77e51f P 264eb8e4d312471993bb5b9f89e9722f: Deleting consensus metadata
I20260808 00:42:52.428073 32189 catalog_manager.cc:4841] TS 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925): tablet 121a80e541634a6f9b178486fc77e51f (table test-table [id=96cb719b0c91415d9c5bf8ab2d4ea1c7]) successfully deleted
I20260808 00:42:52.428529 500 tablet_bootstrap.cc:492] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: No bootstrap required, opened a new log
I20260808 00:42:52.428581 500 ts_tablet_manager.cc:1397] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 00:42:52.428763 500 raft_consensus.cc:348] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.428843 500 raft_consensus.cc:374] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:52.428861 500 raft_consensus.cc:729] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264eb8e4d312471993bb5b9f89e9722f, State: Initialized, Role: FOLLOWER
I20260808 00:42:52.428905 500 consensus_queue.cc:260] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.429005 500 ts_tablet_manager.cc:1428] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:52.493162 569 raft_consensus.cc:482] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:42:52.493314 569 raft_consensus.cc:504] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.493561 569 leader_election.cc:288] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409), 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:52.493986 384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cd527462504e8b9e618fea705a4b99" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f" is_pre_election: true
I20260808 00:42:52.494119 384 raft_consensus.cc:2454] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 0.
I20260808 00:42:52.494339 32339 leader_election.cc:302] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 264eb8e4d312471993bb5b9f89e9722f, eb35618b1ef44514ab63594cc2921b4a; no voters:
I20260808 00:42:52.494655 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cd527462504e8b9e618fea705a4b99" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" is_pre_election: true
I20260808 00:42:52.494772 32640 raft_consensus.cc:2454] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 0.
I20260808 00:42:52.494886 569 raft_consensus.cc:2781] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:42:52.494935 569 raft_consensus.cc:482] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:42:52.494971 569 raft_consensus.cc:3037] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.496331 569 raft_consensus.cc:504] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.496548 569 leader_election.cc:288] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 election: Requested vote from peers 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409), 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925)
I20260808 00:42:52.496771 32640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cd527462504e8b9e618fea705a4b99" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d13faf37abd4b0c8a108fee6854e0e4"
I20260808 00:42:52.496868 32640 raft_consensus.cc:3037] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.496812 384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41cd527462504e8b9e618fea705a4b99" candidate_uuid: "eb35618b1ef44514ab63594cc2921b4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264eb8e4d312471993bb5b9f89e9722f"
I20260808 00:42:52.496915 384 raft_consensus.cc:3037] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:52.497862 32640 raft_consensus.cc:2454] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 1.
I20260808 00:42:52.497975 384 raft_consensus.cc:2454] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb35618b1ef44514ab63594cc2921b4a in term 1.
I20260808 00:42:52.498090 32336 leader_election.cc:302] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d13faf37abd4b0c8a108fee6854e0e4, eb35618b1ef44514ab63594cc2921b4a; no voters:
I20260808 00:42:52.498199 569 raft_consensus.cc:2781] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:52.498250 569 raft_consensus.cc:686] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [term 1 LEADER]: Becoming Leader. State: Replica: eb35618b1ef44514ab63594cc2921b4a, State: Running, Role: LEADER
I20260808 00:42:52.498303 569 consensus_queue.cc:237] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } }
I20260808 00:42:52.499109 32204 catalog_manager.cc:5495] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a reported cstate change: term changed from 0 to 1, leader changed from <none> to eb35618b1ef44514ab63594cc2921b4a (127.20.105.65). New cstate: current_term: 1 leader_uuid: "eb35618b1ef44514ab63594cc2921b4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb35618b1ef44514ab63594cc2921b4a" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46127 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 } health_report { overall_health: UNKNOWN } } }
I20260808 00:42:52.527447 493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.524221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60562 (local address 127.20.105.65:46127)
0808 00:42:52.524274 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:52.524276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:52.524286 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:42:52.524314 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:52.524315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:52.524330 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:52.524394 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:42:52.524605 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.525401 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.526525 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:52.526827 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:52.526914 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:52.527117 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:42:52.527163 (+ 46us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:42:52.527224 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:52.527307 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 00:42:52.527333 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:42:52.530838 384 raft_consensus.cc:1261] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Refusing update from remote peer eb35618b1ef44514ab63594cc2921b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.531010 32639 raft_consensus.cc:1261] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Refusing update from remote peer eb35618b1ef44514ab63594cc2921b4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:42:52.531261 544 consensus_queue.cc:1035] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "264eb8e4d312471993bb5b9f89e9722f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 38925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.531383 504 consensus_queue.cc:1035] T 41cd527462504e8b9e618fea705a4b99 P eb35618b1ef44514ab63594cc2921b4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d13faf37abd4b0c8a108fee6854e0e4" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 33409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:42:52.552309 32206 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:52.552496 32206 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:42:52.553431 32206 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85e7de31fd01456e9037e9709ac2656a: Sending DeleteTablet for 3 replicas of tablet 41cd527462504e8b9e618fea705a4b99
I20260808 00:42:52.553813 32610 tablet_service.cc:1514] Processing DeleteTablet for tablet 41cd527462504e8b9e618fea705a4b99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37432
I20260808 00:42:52.553956 539 tablet_replica.cc:330] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: stopping tablet replica
I20260808 00:42:52.554042 539 raft_consensus.cc:2229] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:52.554093 539 raft_consensus.cc:2258] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.554737 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 32308
I20260808 00:42:52.569731 337 tablet_service.cc:1514] Processing DeleteTablet for tablet 41cd527462504e8b9e618fea705a4b99 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:42:52 UTC) from {username='oryx', principal='oryx/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37508
I20260808 00:42:52.570127 537 tablet_replica.cc:330] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: stopping tablet replica
I20260808 00:42:52.570223 537 raft_consensus.cc:2229] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:42:52.570284 537 raft_consensus.cc:2258] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:42:52.572724 537 ts_tablet_manager.cc:1905] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.572793 539 ts_tablet_manager.cc:1905] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:42:52.574901 539 ts_tablet_manager.cc:1918] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:52.574980 539 log.cc:1196] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/41cd527462504e8b9e618fea705a4b99
I20260808 00:42:52.575003 537 ts_tablet_manager.cc:1918] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:42:52.575068 537 log.cc:1196] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/41cd527462504e8b9e618fea705a4b99
I20260808 00:42:52.575237 539 ts_tablet_manager.cc:1939] T 41cd527462504e8b9e618fea705a4b99 P 9d13faf37abd4b0c8a108fee6854e0e4: Deleting consensus metadata
I20260808 00:42:52.575302 537 ts_tablet_manager.cc:1939] T 41cd527462504e8b9e618fea705a4b99 P 264eb8e4d312471993bb5b9f89e9722f: Deleting consensus metadata
I20260808 00:42:52.575755 32186 catalog_manager.cc:4841] TS 9d13faf37abd4b0c8a108fee6854e0e4 (127.20.105.66:33409): tablet 41cd527462504e8b9e618fea705a4b99 (table test-table [id=ea9de6671ff6447cb9a4aa5cd33f6bec]) successfully deleted
I20260808 00:42:52.575980 32189 catalog_manager.cc:4841] TS 264eb8e4d312471993bb5b9f89e9722f (127.20.105.67:38925): tablet 41cd527462504e8b9e618fea705a4b99 (table test-table [id=ea9de6671ff6447cb9a4aa5cd33f6bec]) successfully deleted
W20260808 00:42:52.582144 32186 connection.cc:535] client connection to 127.20.105.65:46127 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 00:42:52.582355 32186 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:42:52.582650 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 32512
I20260808 00:42:52.586686 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:52.582825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46127 (local address 127.0.0.1:60572)
0808 00:42:52.586494 (+ 3669us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:52.586585 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3636,"thread_start_us":57,"threads_started":1}
W20260808 00:42:52.586901 32186 catalog_manager.cc:4584] TS eb35618b1ef44514ab63594cc2921b4a (127.20.105.65:46127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41cd527462504e8b9e618fea705a4b99: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46127: connect: Connection refused (error 111)
I20260808 00:42:52.599419 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 32721
I20260808 00:42:52.618340 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 32160
2026-08-08T00:42:52Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3801 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:52 dist-test-slave-kvfs krb5kdc[627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:52 dist-test-slave-kvfs krb5kdc[627](info): set up 2 sockets
Aug 08 00:42:52 dist-test-slave-kvfs krb5kdc[627](info): commencing operation
krb5kdc: starting...
W20260808 00:42:53.899410 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:54 dist-test-slave-kvfs krb5kdc[627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:54.056228 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:42153
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:42153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:54.155423 647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:54.155769 647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:54.155814 647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:54.157186 647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:54.157279 647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:54.157303 647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:54.157326 647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:54.157353 647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:54.159137 647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:42153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:42153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:54.159485 647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:54.159780 647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:54.166019 655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:54.166178 647 server_base.cc:1034] running on GCE node
W20260808 00:42:54.166024 654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:54.166061 657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:54.166853 647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:54.167311 647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:54.168509 647 hybrid_clock.cc:648] HybridClock initialized: now 1786149774168495 us; error 56 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T00:42:54Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1559 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:42:54 dist-test-slave-kvfs krb5kdc[664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:42:54 dist-test-slave-kvfs krb5kdc[664](info): set up 2 sockets
Aug 08 00:42:54 dist-test-slave-kvfs krb5kdc[664](info): commencing operation
krb5kdc: starting...
W20260808 00:42:55.484766 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:42:55 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:42:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:42:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:42:55.640153 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33269
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:33269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:55.741652 685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:55.741992 685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:55.742022 685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:55.743413 685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:55.743468 685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:55.743484 685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:55.743496 685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:55.743510 685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:55.745319 685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:33269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:55.745680 685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:55.746004 685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:55.752323 704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:55.752445 690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:55.752341 695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:55.752616 685 server_base.cc:1034] running on GCE node
I20260808 00:42:55.752974 685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:55.753368 685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:55.754554 685 hybrid_clock.cc:648] HybridClock initialized: now 1786149775754547 us; error 57 us; skew 500 ppm
Aug 08 00:42:55 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:55.757666 685 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:55.758306 685 webserver.cc:460] Webserver started at http://127.20.105.126:40525/ using document root <none> and password file <none>
I20260808 00:42:55.758623 685 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:55.758704 685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:55.758879 685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:55.760370 685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "c7f74ca52a134a4dbef61244f81b29b5"
format_stamp: "Formatted at 2026-08-08 00:42:55 on dist-test-slave-kvfs"
server_key: "92ff96eec0ddfd5aaf40636f6a346fdb"
server_key_iv: "2ea8c6faf5e0ab49946278de507c2781"
server_key_version: "encryptionkey@0"
I20260808 00:42:55.760814 685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "c7f74ca52a134a4dbef61244f81b29b5"
format_stamp: "Formatted at 2026-08-08 00:42:55 on dist-test-slave-kvfs"
server_key: "92ff96eec0ddfd5aaf40636f6a346fdb"
server_key_iv: "2ea8c6faf5e0ab49946278de507c2781"
server_key_version: "encryptionkey@0"
I20260808 00:42:55.762496 685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:55.763556 714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:55.763809 685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:55.763964 685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "c7f74ca52a134a4dbef61244f81b29b5"
format_stamp: "Formatted at 2026-08-08 00:42:55 on dist-test-slave-kvfs"
server_key: "92ff96eec0ddfd5aaf40636f6a346fdb"
server_key_iv: "2ea8c6faf5e0ab49946278de507c2781"
server_key_version: "encryptionkey@0"
I20260808 00:42:55.764094 685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:55.778638 685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:55.781670 685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:55.781840 685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:55.787181 685 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:33269
I20260808 00:42:55.787254 781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:33269 every 8 connection(s)
I20260808 00:42:55.787755 685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:55.788930 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 685
I20260808 00:42:55.789098 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:42:55.789191 782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:42:55.789367 20901 external_mini_cluster.cc:1428] Setting key b8d5bcc4eaf7d770856a4945401e45f1
I20260808 00:42:55.793205 782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Bootstrap starting.
I20260808 00:42:55.794335 782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Neither blocks nor log segments found. Creating new log.
I20260808 00:42:55.794732 782 log.cc:824] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:55.795655 782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: No bootstrap required, opened a new log
Aug 08 00:42:55 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:55.798084 782 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } }
I20260808 00:42:55.798274 782 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:42:55.798300 782 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f74ca52a134a4dbef61244f81b29b5, State: Initialized, Role: FOLLOWER
I20260808 00:42:55.798481 782 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } }
I20260808 00:42:55.798606 782 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:55.798646 782 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:55.798700 782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:42:55.799558 782 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } }
I20260808 00:42:55.799674 782 leader_election.cc:302] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7f74ca52a134a4dbef61244f81b29b5; no voters:
I20260808 00:42:55.799932 782 leader_election.cc:288] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:42:55.799969 789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:42:55.800283 789 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 1 LEADER]: Becoming Leader. State: Replica: c7f74ca52a134a4dbef61244f81b29b5, State: Running, Role: LEADER
I20260808 00:42:55.800397 789 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } }
I20260808 00:42:55.800446 782 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:55.800803 789 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7f74ca52a134a4dbef61244f81b29b5. Latest consensus state: current_term: 1 leader_uuid: "c7f74ca52a134a4dbef61244f81b29b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } } }
I20260808 00:42:55.800879 789 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:55.801054 790 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7f74ca52a134a4dbef61244f81b29b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } } }
I20260808 00:42:55.801151 790 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: This master's current role is: LEADER
W20260808 00:42:55.802778 807 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:42:55.802923 807 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:42:55.803066 795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:55.803103 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:55.790074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41748 (local address 127.20.105.126:33269)
0808 00:42:55.790293 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:55.790310 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:55.790341 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:42:55.791018 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:55.791028 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:55.791088 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:55.791242 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:55.792200 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:55.793245 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:55.794458 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:55.795033 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:55.795189 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:55.798052 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:55.798057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:55.798057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:55.798072 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:55.800703 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:55.801252 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:55.801255 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:55.801256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:55.801383 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:55.801686 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:55.801687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:55.801688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:55.802205 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:55.802427 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:55.802618 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 00:42:55.802794 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":104,"threads_started":1}
I20260808 00:42:55.803309 795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:55.805330 795 catalog_manager.cc:1347] Generated new cluster ID: a49fd1c8f16943aeb3214808558d2da6
I20260808 00:42:55.805395 795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:55.822631 795 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:42:55.823589 795 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:55.829779 795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Generated new TSK 0
I20260808 00:42:55.830101 795 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:42:55.867879 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33269
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:55.975363 814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:55.975701 814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:55.975736 814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:55.977371 814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:55.977437 814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:55.977473 814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:55.979259 814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:55.979622 814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:55.979936 814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:55.986183 824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:55.986209 823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:55.986193 827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:55.986790 814 server_base.cc:1034] running on GCE node
I20260808 00:42:55.987059 814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:55.987329 814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:55.988543 814 hybrid_clock.cc:648] HybridClock initialized: now 1786149775988523 us; error 60 us; skew 500 ppm
Aug 08 00:42:55 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:55.991921 814 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:42:55.992514 814 webserver.cc:460] Webserver started at http://127.20.105.65:35735/ using document root <none> and password file <none>
I20260808 00:42:55.992813 814 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:55.992890 814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:55.993074 814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:55.994339 814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "c2b269bcddd54cb78599441b01757147"
format_stamp: "Formatted at 2026-08-08 00:42:55 on dist-test-slave-kvfs"
server_key: "303a934afefbe9067b206681ad68b417"
server_key_iv: "48d0787435c492efa01b0ca9da361182"
server_key_version: "encryptionkey@0"
I20260808 00:42:55.994769 814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "c2b269bcddd54cb78599441b01757147"
format_stamp: "Formatted at 2026-08-08 00:42:55 on dist-test-slave-kvfs"
server_key: "303a934afefbe9067b206681ad68b417"
server_key_iv: "48d0787435c492efa01b0ca9da361182"
server_key_version: "encryptionkey@0"
I20260808 00:42:55.996383 814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:42:55.997253 837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:55.997794 814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:55.997944 814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "c2b269bcddd54cb78599441b01757147"
format_stamp: "Formatted at 2026-08-08 00:42:55 on dist-test-slave-kvfs"
server_key: "303a934afefbe9067b206681ad68b417"
server_key_iv: "48d0787435c492efa01b0ca9da361182"
server_key_version: "encryptionkey@0"
I20260808 00:42:55.998044 814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:56.024505 814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:56.027415 814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:56.027648 814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:56.027966 814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:56.028412 814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:56.028472 814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.028522 814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:56.028560 814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.036469 814 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:39885
I20260808 00:42:56.036531 998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:39885 every 8 connection(s)
I20260808 00:42:56.037043 814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149775, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:56.047204 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 814
I20260808 00:42:56.047374 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:42:56.047638 20901 external_mini_cluster.cc:1428] Setting key 1a10b960d4d1c32c510a4cab87429e3d
I20260808 00:42:56.048753 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.038084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:58031 (local address 127.20.105.126:33269)
0808 00:42:56.038275 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:56.038278 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:56.038392 (+ 114us) server_negotiation.cc:407] Connection header received
0808 00:42:56.039057 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:56.039059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:56.039074 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:56.039123 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:56.040413 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.041017 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.042148 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.042403 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.042511 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.045107 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:56.045109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:56.045109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:56.045120 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:56.046738 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:56.047242 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:56.047243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:56.047244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:56.047316 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:56.047670 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:56.047671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:56.047671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:56.047892 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:56.047979 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:56.048516 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 00:42:56.048655 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260808 00:42:56.049021 1003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.038045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:58031)
0808 00:42:56.038243 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:56.038295 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:56.038917 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:56.039214 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:56.039220 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:56.039942 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:56.040294 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.040301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.041138 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.041139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.042045 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.042048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.042523 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.042525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.042685 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.043292 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:56.043302 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:56.044949 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:56.046861 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:56.046871 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:56.046874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:56.047151 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:56.047391 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:56.047393 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:56.047393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:56.047559 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:56.048032 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:56.048076 (+ 44us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:56.048279 (+ 203us) client_negotiation.cc:769] Sending connection context
0808 00:42:56.048444 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 00:42:56.048695 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260808 00:42:56.049662 999 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33269
I20260808 00:42:56.049805 999 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:56.050045 999 heartbeater.cc:511] Master 127.20.105.126:33269 requested a full tablet report, sending...
I20260808 00:42:56.050741 732 ts_manager.cc:194] Registered new tserver with Master: c2b269bcddd54cb78599441b01757147 (127.20.105.65:39885)
I20260808 00:42:56.051774 732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:58031
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:42:56.084992 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33269
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:56.194850 1008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:56.195145 1008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:56.195173 1008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:56.196529 1008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:56.196580 1008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:56.196609 1008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:56.198176 1008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:56.198442 1008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:56.198681 1008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:56.205017 1015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:56.204979 1019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:56.205117 1008 server_base.cc:1034] running on GCE node
W20260808 00:42:56.205252 1017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:56.205575 1008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:56.205960 1008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:56.207154 1008 hybrid_clock.cc:648] HybridClock initialized: now 1786149776207129 us; error 48 us; skew 500 ppm
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:56.210511 1008 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:42:56.211146 1008 webserver.cc:460] Webserver started at http://127.20.105.66:46857/ using document root <none> and password file <none>
I20260808 00:42:56.211474 1008 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:56.211562 1008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:56.211759 1008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:56.213276 1008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "e9c471fc44204862b79ec6e9f3c6bb82"
format_stamp: "Formatted at 2026-08-08 00:42:56 on dist-test-slave-kvfs"
server_key: "158fb03537675e034f4fa517b39dddd7"
server_key_iv: "bdd06a05fb00ba304ad5b2b90cb354ec"
server_key_version: "encryptionkey@0"
I20260808 00:42:56.213768 1008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "e9c471fc44204862b79ec6e9f3c6bb82"
format_stamp: "Formatted at 2026-08-08 00:42:56 on dist-test-slave-kvfs"
server_key: "158fb03537675e034f4fa517b39dddd7"
server_key_iv: "bdd06a05fb00ba304ad5b2b90cb354ec"
server_key_version: "encryptionkey@0"
I20260808 00:42:56.215564 1008 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 00:42:56.216660 1030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.216926 1008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:56.217065 1008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "e9c471fc44204862b79ec6e9f3c6bb82"
format_stamp: "Formatted at 2026-08-08 00:42:56 on dist-test-slave-kvfs"
server_key: "158fb03537675e034f4fa517b39dddd7"
server_key_iv: "bdd06a05fb00ba304ad5b2b90cb354ec"
server_key_version: "encryptionkey@0"
I20260808 00:42:56.217160 1008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:56.234862 1008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:56.237787 1008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:56.238003 1008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:56.238332 1008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:56.238924 1008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:56.239010 1008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.239125 1008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:56.239166 1008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.247192 1008 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:32811
I20260808 00:42:56.247241 1193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:32811 every 8 connection(s)
I20260808 00:42:56.247816 1008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:42:56.253757 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 1008
I20260808 00:42:56.253875 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:42:56.254127 20901 external_mini_cluster.cc:1428] Setting key 3fa59a1f1d4d742965658f3d99b7f7fd
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:56.260994 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.248838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:44139 (local address 127.20.105.126:33269)
0808 00:42:56.248929 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:56.248932 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:56.249272 (+ 340us) server_negotiation.cc:407] Connection header received
0808 00:42:56.249963 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:56.249965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:56.249978 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:56.250051 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:56.251355 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.251952 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.253154 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.253392 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.253541 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.256831 (+ 3290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:56.256833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:56.256833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:56.256845 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:56.258977 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:56.259651 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:56.259652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:56.259652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:56.259724 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:56.260128 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:56.260129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:56.260130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:56.260279 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:56.260345 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:56.260787 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 00:42:56.260896 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:42:56.261363 1196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.248810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:44139)
0808 00:42:56.249066 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:56.249120 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:56.249795 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:56.250217 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:56.250222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:56.250944 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:56.251212 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.251218 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.252081 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.252081 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.253045 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.253049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.253575 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.253577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.253742 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.254554 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:56.254565 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:56.256684 (+ 2119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:56.259088 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:56.259099 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:56.259103 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:56.259545 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:56.259808 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:56.259809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:56.259809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:56.260011 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:56.260357 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:56.260366 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:56.260602 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 00:42:56.260739 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 00:42:56.260984 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":118,"threads_started":1}
I20260808 00:42:56.262187 1194 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33269
I20260808 00:42:56.262347 1194 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:56.262642 1194 heartbeater.cc:511] Master 127.20.105.126:33269 requested a full tablet report, sending...
I20260808 00:42:56.263221 732 ts_manager.cc:194] Registered new tserver with Master: e9c471fc44204862b79ec6e9f3c6bb82 (127.20.105.66:32811)
I20260808 00:42:56.263921 732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:44139
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:42:56.292480 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33269
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:56.394825 1201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:56.395119 1201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:56.395147 1201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:56.396937 1201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:56.396991 1201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:56.397020 1201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:56.398599 1201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:56.398869 1201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:56.399106 1201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:56.405471 1208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:56.405479 1209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:56.405484 1211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:56.406003 1201 server_base.cc:1034] running on GCE node
I20260808 00:42:56.406286 1201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:56.406599 1201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:56.407799 1201 hybrid_clock.cc:648] HybridClock initialized: now 1786149776407774 us; error 60 us; skew 500 ppm
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:56.411024 1201 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:42:56.411614 1201 webserver.cc:460] Webserver started at http://127.20.105.67:42269/ using document root <none> and password file <none>
I20260808 00:42:56.411890 1201 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:56.411942 1201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:56.412146 1201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:42:56.413458 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "b035f75d04cb464e9c89c68af3cd5ea8"
format_stamp: "Formatted at 2026-08-08 00:42:56 on dist-test-slave-kvfs"
server_key: "390227f2d6d97bdde89bb7144440590b"
server_key_iv: "10fb7c6f48a66b8ee0bb910c90065e90"
server_key_version: "encryptionkey@0"
I20260808 00:42:56.413904 1201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "b035f75d04cb464e9c89c68af3cd5ea8"
format_stamp: "Formatted at 2026-08-08 00:42:56 on dist-test-slave-kvfs"
server_key: "390227f2d6d97bdde89bb7144440590b"
server_key_iv: "10fb7c6f48a66b8ee0bb910c90065e90"
server_key_version: "encryptionkey@0"
I20260808 00:42:56.415503 1201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:42:56.416669 1220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.416935 1201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:42:56.417048 1201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "b035f75d04cb464e9c89c68af3cd5ea8"
format_stamp: "Formatted at 2026-08-08 00:42:56 on dist-test-slave-kvfs"
server_key: "390227f2d6d97bdde89bb7144440590b"
server_key_iv: "10fb7c6f48a66b8ee0bb910c90065e90"
server_key_version: "encryptionkey@0"
I20260808 00:42:56.417131 1201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:56.432358 1201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:56.435195 1201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:56.435379 1201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:56.435725 1201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:56.436184 1201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:56.436244 1201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.436288 1201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:56.436317 1201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.443521 1201 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:36357
I20260808 00:42:56.443581 1390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:36357 every 8 connection(s)
I20260808 00:42:56.443982 1201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:42:56.450917 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 1201
I20260808 00:42:56.451062 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:42:56.451303 20901 external_mini_cluster.cc:1428] Setting key 13280dd8fcf351f7c2b19d3e6e6a7321
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:42:56.456126 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.444964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:39957 (local address 127.20.105.126:33269)
0808 00:42:56.445045 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:56.445048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:56.445302 (+ 254us) server_negotiation.cc:407] Connection header received
0808 00:42:56.445937 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:56.445939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:56.445953 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:56.446012 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:56.447131 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.447753 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.448978 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.449208 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.449281 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.452576 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:56.452578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:56.452578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:56.452594 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:56.454299 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:56.454830 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:56.454832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:56.454832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:56.454910 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:56.455245 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:56.455246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:56.455247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:56.455404 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:56.455489 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:56.455857 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 00:42:56.455962 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:42:56.456411 1393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.444943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:39957)
0808 00:42:56.445160 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:56.445204 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:56.445797 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:56.446081 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:56.446084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:56.446764 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:56.447016 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.447021 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.447859 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.447860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.448839 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.448843 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.449351 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.449353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.449500 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.450279 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:56.450290 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:56.452448 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:42:56.454424 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:56.454434 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:56.454437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:56.454734 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:56.454999 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:42:56.455000 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:42:56.455000 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:42:56.455157 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:42:56.455492 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:42:56.455501 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:42:56.455686 (+ 185us) client_negotiation.cc:769] Sending connection context
0808 00:42:56.455817 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:42:56.456088 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":125,"threads_started":1}
I20260808 00:42:56.457087 1391 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33269
I20260808 00:42:56.457206 1391 heartbeater.cc:464] Registering TS with master...
I20260808 00:42:56.457468 1391 heartbeater.cc:511] Master 127.20.105.126:33269 requested a full tablet report, sending...
I20260808 00:42:56.457989 732 ts_manager.cc:194] Registered new tserver with Master: b035f75d04cb464e9c89c68af3cd5ea8 (127.20.105.67:36357)
I20260808 00:42:56.458493 732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:39957
I20260808 00:42:56.463701 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab.
Entry for principal oryx/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab.
WARNING: no policy specified for oryx/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab.
Entry for principal oryx/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab.
WARNING: no policy specified for oryx/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab.
Entry for principal oryx/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab.
I20260808 00:42:56.564333 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 814
I20260808 00:42:56.574560 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 1008
I20260808 00:42:56.589128 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 1201
I20260808 00:42:56.608340 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 685
I20260808 00:42:56.625442 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33269
--webserver_interface=127.20.105.126
--webserver_port=40525
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:33269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:42:56.722429 1402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:56.722718 1402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:56.722744 1402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:56.724144 1402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:42:56.724200 1402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:56.724215 1402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:56.724229 1402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:42:56.724241 1402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:42:56.725977 1402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:33269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=40525
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:56.726318 1402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:56.726616 1402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:56.732738 1407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:56.732740 1412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:56.732781 1408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:56.732935 1402 server_base.cc:1034] running on GCE node
I20260808 00:42:56.733417 1402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:56.733834 1402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:56.735025 1402 hybrid_clock.cc:648] HybridClock initialized: now 1786149776735007 us; error 52 us; skew 500 ppm
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149776, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:56.738087 1402 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:42:56.738615 1402 webserver.cc:460] Webserver started at http://127.20.105.126:40525/ using document root <none> and password file <none>
I20260808 00:42:56.738893 1402 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:56.738998 1402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:56.740736 1402 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:42:56.741639 1423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.741955 1402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:56.742134 1402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "c7f74ca52a134a4dbef61244f81b29b5"
format_stamp: "Formatted at 2026-08-08 00:42:55 on dist-test-slave-kvfs"
server_key: "92ff96eec0ddfd5aaf40636f6a346fdb"
server_key_iv: "2ea8c6faf5e0ab49946278de507c2781"
server_key_version: "encryptionkey@0"
I20260808 00:42:56.742756 1402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:56.753587 1402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:56.756003 1402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:56.756289 1402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:56.761854 1402 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:33269
I20260808 00:42:56.761909 1504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:33269 every 8 connection(s)
I20260808 00:42:56.762480 1402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:42:56.762748 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 1402
I20260808 00:42:56.763124 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:39885
--local_ip_for_outbound_sockets=127.20.105.65
--tserver_master_addrs=127.20.105.126:33269
--webserver_port=35735
--webserver_interface=127.20.105.65
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.105.65
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:42:56.764762 1506 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:42:56.765637 1506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Bootstrap starting.
I20260808 00:42:56.780552 1506 log.cc:824] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:42:56.783785 1506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:42:56.784247 1506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Bootstrap complete.
I20260808 00:42:56.788467 1506 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } }
I20260808 00:42:56.788877 1506 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7f74ca52a134a4dbef61244f81b29b5, State: Initialized, Role: FOLLOWER
I20260808 00:42:56.789093 1506 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } }
I20260808 00:42:56.789175 1506 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:42:56.789220 1506 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:42:56.789275 1506 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:42:56.790988 1506 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } }
I20260808 00:42:56.791128 1506 leader_election.cc:302] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7f74ca52a134a4dbef61244f81b29b5; no voters:
I20260808 00:42:56.791361 1506 leader_election.cc:288] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:42:56.791599 1513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:42:56.791867 1513 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [term 2 LEADER]: Becoming Leader. State: Replica: c7f74ca52a134a4dbef61244f81b29b5, State: Running, Role: LEADER
I20260808 00:42:56.791872 1506 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:42:56.792124 1513 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } }
I20260808 00:42:56.792532 1514 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c7f74ca52a134a4dbef61244f81b29b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } } }
I20260808 00:42:56.792640 1514 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:56.793109 1525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:42:56.793315 1515 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7f74ca52a134a4dbef61244f81b29b5. Latest consensus state: current_term: 2 leader_uuid: "c7f74ca52a134a4dbef61244f81b29b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7f74ca52a134a4dbef61244f81b29b5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33269 } } }
I20260808 00:42:56.793428 1515 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:42:56.794148 1525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:42:56.794390 1525 catalog_manager.cc:1259] Loaded cluster ID: a49fd1c8f16943aeb3214808558d2da6
I20260808 00:42:56.794607 1525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:42:56.795586 1525 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:42:56.795840 1525 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c7f74ca52a134a4dbef61244f81b29b5: Loaded TSK: 0
I20260808 00:42:56.796166 1525 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:42:56.883447 1510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:56.883770 1510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:56.883816 1510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:56.885191 1510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:56.885268 1510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:56.885318 1510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:42:56.886909 1510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.65.keytab
--principal=oryx/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:39885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=35735
--tserver_master_addrs=127.20.105.126:33269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:56.887240 1510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:56.887534 1510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:56.893635 1548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:56.893678 1544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:56.893818 1551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:56.894155 1510 server_base.cc:1034] running on GCE node
I20260808 00:42:56.894421 1510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:56.894728 1510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:56.895906 1510 hybrid_clock.cc:648] HybridClock initialized: now 1786149776895879 us; error 47 us; skew 500 ppm
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149776, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:56.898942 1510 init.cc:377] Logged in from keytab as oryx/127.20.105.65@KRBTEST.COM (short username oryx)
I20260808 00:42:56.899538 1510 webserver.cc:460] Webserver started at http://127.20.105.65:35735/ using document root <none> and password file <none>
I20260808 00:42:56.899835 1510 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:56.899904 1510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:56.901589 1510 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:56.902326 1563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.902554 1510 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:42:56.902661 1510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "c2b269bcddd54cb78599441b01757147"
format_stamp: "Formatted at 2026-08-08 00:42:55 on dist-test-slave-kvfs"
server_key: "303a934afefbe9067b206681ad68b417"
server_key_iv: "48d0787435c492efa01b0ca9da361182"
server_key_version: "encryptionkey@0"
I20260808 00:42:56.903116 1510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:56.922049 1510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:56.924980 1510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:56.925195 1510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:56.925541 1510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:56.925957 1510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:56.926019 1510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.926069 1510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:56.926093 1510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:56.934887 1510 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:39885
I20260808 00:42:56.934979 1723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:39885 every 8 connection(s)
I20260808 00:42:56.935431 1510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:42:56.944614 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 1510
I20260808 00:42:56.945036 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:32811
--local_ip_for_outbound_sockets=127.20.105.66
--tserver_master_addrs=127.20.105.126:33269
--webserver_port=46857
--webserver_interface=127.20.105.66
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.105.66
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:56.947248 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.936588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:46573)
0808 00:42:56.936825 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:56.936882 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:56.937533 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:56.938093 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:56.938097 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:56.938801 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:56.939064 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.939070 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.941063 (+ 1993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.941064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.941972 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.941975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.943007 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.943008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.943141 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.943875 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:56.943887 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:56.946915 (+ 3028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":111,"threads_started":1}
W20260808 00:42:56.947265 1730 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:56.936641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:46573 (local address 127.20.105.126:33269)
0808 00:42:56.936963 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:56.936975 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:56.937039 (+ 64us) server_negotiation.cc:407] Connection header received
0808 00:42:56.937803 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:56.937811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:56.937878 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:56.938051 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:56.939199 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.940927 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.942072 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.942832 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.942987 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.947096 (+ 4109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:46573: BlockingRecv error: recv got EOF from 127.20.105.65:46573 (error 108)
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":116,"threads_started":1}
W20260808 00:42:56.947983 1725 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:33269 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.105.126:33269: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:56.954485 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.948312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:50331)
0808 00:42:56.949629 (+ 1317us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:56.949642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:56.949721 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:56.949888 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:56.949889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:56.950060 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:56.950126 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.950128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.950843 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.950844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.951391 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.951393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.951789 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.951790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.951865 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.952427 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:56.952433 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:56.954381 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113}
W20260808 00:42:56.954553 1730 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:56.948376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:50331 (local address 127.20.105.126:33269)
0808 00:42:56.948438 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:56.948440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:56.949701 (+ 1261us) server_negotiation.cc:407] Connection header received
0808 00:42:56.949792 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:56.949794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:56.949810 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:56.949885 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:56.950221 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.950753 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.951494 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.951702 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.951779 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.954476 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:50331: BlockingRecv error: recv got EOF from 127.20.105.65:50331 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:56 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:56.959381 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:56.954863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:38745)
0808 00:42:56.954948 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:56.954960 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:56.955011 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:56.955153 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:56.955155 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:56.955290 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:56.955325 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.955327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.955998 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.955999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.956572 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:56.956574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.956966 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.956967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:56.957032 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.957524 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:56.957529 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:56.959280 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 00:42:56.959434 1730 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:56.954908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:38745 (local address 127.20.105.126:33269)
0808 00:42:56.954950 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:56.954952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:56.954993 (+ 41us) server_negotiation.cc:407] Connection header received
0808 00:42:56.955083 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:56.955084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:56.955095 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:56.955141 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:56.955409 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.955906 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.956672 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:56.956877 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:56.956965 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:56.959361 (+ 2396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:38745: BlockingRecv error: recv got EOF from 127.20.105.65:38745 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 00:42:56.959580 1725 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:42:57.048936 1731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:57.049286 1731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:57.049325 1731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:57.050798 1731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:57.050849 1731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:57.050887 1731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:42:57.052507 1731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.66.keytab
--principal=oryx/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:32811
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=46857
--tserver_master_addrs=127.20.105.126:33269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:57.052757 1731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:57.053004 1731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:57.059303 1740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:57.059485 1731 server_base.cc:1034] running on GCE node
W20260808 00:42:57.059305 1743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:57.059324 1741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:57.059981 1731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:57.060359 1731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:57.061514 1731 hybrid_clock.cc:648] HybridClock initialized: now 1786149777061491 us; error 40 us; skew 500 ppm
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149777, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:57.064894 1731 init.cc:377] Logged in from keytab as oryx/127.20.105.66@KRBTEST.COM (short username oryx)
I20260808 00:42:57.065433 1731 webserver.cc:460] Webserver started at http://127.20.105.66:46857/ using document root <none> and password file <none>
I20260808 00:42:57.065671 1731 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:57.065739 1731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:57.067373 1731 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:57.068166 1752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:57.068373 1731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:57.068492 1731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "e9c471fc44204862b79ec6e9f3c6bb82"
format_stamp: "Formatted at 2026-08-08 00:42:56 on dist-test-slave-kvfs"
server_key: "158fb03537675e034f4fa517b39dddd7"
server_key_iv: "bdd06a05fb00ba304ad5b2b90cb354ec"
server_key_version: "encryptionkey@0"
I20260808 00:42:57.068921 1731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:57.084273 1731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:57.086862 1731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:57.087049 1731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:57.087373 1731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:57.087775 1731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:57.087812 1731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:57.087839 1731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:57.087891 1731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:57.095098 1731 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:32811
I20260808 00:42:57.095224 1913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:32811 every 8 connection(s)
I20260808 00:42:57.095625 1731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:57.104419 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 1731
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:57.104833 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:36357
--local_ip_for_outbound_sockets=127.20.105.67
--tserver_master_addrs=127.20.105.126:33269
--webserver_port=42269
--webserver_interface=127.20.105.67
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.105.67
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:42:57.105978 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:57.096656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:53873)
0808 00:42:57.096934 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:57.096985 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:57.097649 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:57.097934 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:57.097939 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:57.098747 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:57.099045 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.099052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.099946 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.099947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.100993 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.100997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.101417 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.101417 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.101543 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.102201 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:57.102213 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:57.105743 (+ 3530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":127,"threads_started":1}
W20260808 00:42:57.106103 1730 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:57.096690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:53873 (local address 127.20.105.126:33269)
0808 00:42:57.096784 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:57.096787 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:57.097096 (+ 309us) server_negotiation.cc:407] Connection header received
0808 00:42:57.097776 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:57.097777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:57.097794 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:57.097857 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:57.099180 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.099828 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.101116 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.101328 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.101399 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.105987 (+ 4588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:53873: BlockingRecv error: recv got EOF from 127.20.105.66:53873 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 00:42:57.106520 1914 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:33269 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.105.126:33269: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:57.112197 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:57.106853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:53045)
0808 00:42:57.106917 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:57.106931 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:57.106992 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:57.107155 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:57.107156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:57.107353 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:57.107426 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.107433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.108329 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.108330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.108936 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.108937 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.109409 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.109410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.109492 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.110032 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:57.110037 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:57.112097 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 00:42:57.112248 1730 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:57.106934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:53045 (local address 127.20.105.126:33269)
0808 00:42:57.106997 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:57.106999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:57.107011 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:57.107074 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:57.107075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:57.107086 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:57.107150 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:57.107535 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.108238 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.109055 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.109342 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.109414 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.112171 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:53045: BlockingRecv error: recv got EOF from 127.20.105.66:53045 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:57.117259 1919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:57.112644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:45177)
0808 00:42:57.112697 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:57.112709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:57.112755 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:57.112922 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:57.112923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:57.113061 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:57.113095 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.113096 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.113768 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.113769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.114275 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.114276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.114648 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.114649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.114703 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.115284 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:57.115289 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:57.117157 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:42:57.117317 1730 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:57.112744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:45177 (local address 127.20.105.126:33269)
0808 00:42:57.112807 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:57.112810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:57.112819 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:42:57.112843 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:57.112844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:57.112854 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:57.112914 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:57.113185 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.113672 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.114372 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.114574 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.114651 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.117241 (+ 2590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:45177: BlockingRecv error: recv got EOF from 127.20.105.66:45177 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 00:42:57.117578 1914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:42:57.207757 1921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:42:57.208102 1921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:42:57.208164 1921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:42:57.209561 1921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:42:57.209611 1921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:42:57.209645 1921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:42:57.211328 1921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/krb5kdc/oryx_127.20.105.67.keytab
--principal=oryx/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:36357
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=42269
--tserver_master_addrs=127.20.105.126:33269
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:42:57.211655 1921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:42:57.211963 1921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:42:57.218312 1927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:57.218375 1928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:42:57.218375 1931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:42:57.218586 1921 server_base.cc:1034] running on GCE node
I20260808 00:42:57.218803 1921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:42:57.219087 1921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:42:57.220280 1921 hybrid_clock.cc:648] HybridClock initialized: now 1786149777220261 us; error 37 us; skew 500 ppm
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149777, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:42:57.223336 1921 init.cc:377] Logged in from keytab as oryx/127.20.105.67@KRBTEST.COM (short username oryx)
I20260808 00:42:57.223913 1921 webserver.cc:460] Webserver started at http://127.20.105.67:42269/ using document root <none> and password file <none>
I20260808 00:42:57.224223 1921 fs_manager.cc:359] Metadata directory not provided
I20260808 00:42:57.224325 1921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:42:57.225983 1921 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:42:57.226724 1939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:57.226933 1921 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:42:57.227049 1921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "b035f75d04cb464e9c89c68af3cd5ea8"
format_stamp: "Formatted at 2026-08-08 00:42:56 on dist-test-slave-kvfs"
server_key: "390227f2d6d97bdde89bb7144440590b"
server_key_iv: "10fb7c6f48a66b8ee0bb910c90065e90"
server_key_version: "encryptionkey@0"
I20260808 00:42:57.227506 1921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:42:57.253324 1921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:42:57.256637 1921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:42:57.256938 1921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:42:57.257261 1921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:42:57.257683 1921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:42:57.257720 1921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:57.257747 1921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:42:57.257795 1921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:42:57.265067 1921 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:36357
I20260808 00:42:57.265153 2107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:36357 every 8 connection(s)
I20260808 00:42:57.265599 1921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:57.274152 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 1921
W20260808 00:42:57.274669 1730 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:57.266672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:56585 (local address 127.20.105.126:33269)
0808 00:42:57.266781 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:57.266784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:57.267017 (+ 233us) server_negotiation.cc:407] Connection header received
0808 00:42:57.267659 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:57.267660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:57.267674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:57.267735 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:57.268973 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.269543 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.270612 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.270831 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.270915 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.274601 (+ 3686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:56585: BlockingRecv error: recv got EOF from 127.20.105.67:56585 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 00:42:57.274748 2110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:57.266649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:56585)
0808 00:42:57.266871 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:57.266916 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:57.267536 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:57.267809 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:57.267813 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:57.268605 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:57.268858 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.268863 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.269634 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.269635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.270510 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.270513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.270933 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.270934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.271073 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.271740 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:57.271750 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:57.274484 (+ 2734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":132,"threads_started":1}
W20260808 00:42:57.275043 2108 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:33269 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.105.126:33269: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:57.281070 2110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:57.275331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:50037)
0808 00:42:57.275535 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:57.275556 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:57.275607 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:57.275779 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:57.275780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:57.275927 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:57.275979 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.275981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.276693 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.276694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.277236 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.277238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.277663 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.277664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.277744 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.278287 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:57.278293 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:57.280936 (+ 2643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184}
W20260808 00:42:57.281116 2114 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:57.275377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:50037 (local address 127.20.105.126:33269)
0808 00:42:57.275537 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:57.275539 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:57.275590 (+ 51us) server_negotiation.cc:407] Connection header received
0808 00:42:57.275667 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:57.275668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:57.275679 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:57.275776 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:57.276105 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.276605 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.277345 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.277574 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.277656 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.281033 (+ 3377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:50037: BlockingRecv error: recv got EOF from 127.20.105.67:50037 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":114,"threads_started":1}
I20260808 00:42:57.284147 1730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:57.274849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41750 (local address 127.20.105.126:33269)
0808 00:42:57.275170 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:57.275172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:57.275182 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:42:57.275213 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:57.275214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:57.275231 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:57.275284 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:57.276038 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.276582 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.277433 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.277642 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.277734 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.279556 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:42:57.279564 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:42:57.279565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:42:57.279583 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:42:57.282234 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:57.282720 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:57.282726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:57.282727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:57.282871 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:42:57.283127 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:42:57.283128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:42:57.283128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:42:57.283454 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:42:57.283709 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 00:42:57.283807 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 00:42:57.283967 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:57.286371 2110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:57.281552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:53849)
0808 00:42:57.281605 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:57.281618 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:57.281659 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:57.281789 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:57.281790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:57.281929 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:57.281967 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.281969 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.282667 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.282668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.283195 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.283197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.283604 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.283605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.283663 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.284147 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:57.284152 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:57.286308 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:42:57.286516 2114 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:57.281583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:53849 (local address 127.20.105.126:33269)
0808 00:42:57.281629 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:57.281631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:57.281643 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:42:57.281713 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:57.281714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:57.281726 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:57.281785 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:57.282065 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.282581 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.283284 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.283507 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.283603 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.286417 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:53849: BlockingRecv error: recv got EOF from 127.20.105.67:53849 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 00:42:57.286574 2108 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:57 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:57.965740 2115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:57.960175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:55363)
0808 00:42:57.960347 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:57.960361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:57.960425 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:57.960568 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:57.960569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:57.960781 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:57.960854 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.960856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.961587 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.961588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.962185 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:57.962186 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.962625 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.962625 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:57.962672 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.963289 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:57.963294 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:57.965641 (+ 2347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
W20260808 00:42:57.965924 2117 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:57.960273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:55363 (local address 127.20.105.126:33269)
0808 00:42:57.960441 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:57.960443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:57.960451 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:42:57.960478 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:57.960479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:57.960493 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:57.960565 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:57.960924 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.961509 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.962295 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:57.962544 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:57.962612 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:57.965796 (+ 3184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:55363: BlockingRecv error: recv got EOF from 127.20.105.65:55363 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
Aug 08 00:42:58 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:58 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:58.123597 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:58.118118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:49081)
0808 00:42:58.118293 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:58.118314 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:58.118391 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:58.118541 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:58.118542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:58.118778 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:58.118852 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:58.118854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.119606 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.119607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:58.120289 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:58.120292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.120710 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.120710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:58.120756 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:58.121332 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:58.121339 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:58.123492 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
W20260808 00:42:58.123688 2117 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:58.118233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:49081 (local address 127.20.105.126:33269)
0808 00:42:58.118313 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:58.118316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:58.118407 (+ 91us) server_negotiation.cc:407] Connection header received
0808 00:42:58.118442 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:58.118443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:58.118456 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:58.118521 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:58.118945 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.119530 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.120418 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.120632 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.120698 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:58.123628 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:49081: BlockingRecv error: recv got EOF from 127.20.105.66:49081 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 00:42:58 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:58 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:58.293473 2120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:58.287178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:46407)
0808 00:42:58.287453 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:58.287467 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:58.287560 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:58.287809 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:58.287811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:58.288125 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:58.288204 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:58.288207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.289086 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.289088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:58.289754 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:58.289756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.290204 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.290205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:58.290285 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:58.290893 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:58.290900 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:58.293374 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":136,"threads_started":1}
W20260808 00:42:58.293690 2117 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:58.287314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:46407 (local address 127.20.105.126:33269)
0808 00:42:58.287441 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:58.287444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:58.287554 (+ 110us) server_negotiation.cc:407] Connection header received
0808 00:42:58.287648 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:58.287649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:58.287673 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:58.287737 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:58.288321 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.288963 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.289867 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.290104 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.290202 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:58.293533 (+ 3331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:46407: BlockingRecv error: recv got EOF from 127.20.105.67:46407 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 00:42:58 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:58 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:58.972417 2124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:58.966501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:44299)
0808 00:42:58.966707 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:58.966730 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:58.966801 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:58.966989 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:58.966990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:58.967227 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:58.967298 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:58.967299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.968156 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.968157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:58.968760 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:58.968762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.969286 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.969287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:58.969381 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:58.969974 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:58.969980 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:58.972335 (+ 2355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":126,"threads_started":1}
W20260808 00:42:58.972577 2125 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:58.966644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:44299 (local address 127.20.105.126:33269)
0808 00:42:58.966807 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:58.966810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:58.966824 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:42:58.966867 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:58.966867 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:58.966881 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:58.966960 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:58.967372 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.967959 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.968840 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:58.969186 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:58.969262 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:58.972450 (+ 3188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:44299: BlockingRecv error: recv got EOF from 127.20.105.65:44299 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
Aug 08 00:42:59 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:59 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:59.130709 2126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:59.124410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:43693)
0808 00:42:59.124663 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:59.124682 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:59.124815 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:59.125114 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:59.125116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:59.125454 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:59.125541 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:59.125544 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.126406 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.126408 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:59.127040 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:59.127041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.127492 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.127493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:59.127562 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:59.128224 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:59.128230 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:59.130575 (+ 2345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":140,"threads_started":1}
W20260808 00:42:59.130793 2125 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:59.124581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:43693 (local address 127.20.105.126:33269)
0808 00:42:59.124690 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:59.124693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:59.124784 (+ 91us) server_negotiation.cc:407] Connection header received
0808 00:42:59.124992 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:59.124993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:59.125010 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:59.125104 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:59.125651 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.126298 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.127145 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.127389 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.127493 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:59.130713 (+ 3220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:43693: BlockingRecv error: recv got EOF from 127.20.105.66:43693 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 00:42:59 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:59 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:59.300743 2127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:59.294191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:58073)
0808 00:42:59.294335 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:59.294349 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:59.294452 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:59.294725 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:59.294726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:59.294993 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:59.295110 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:59.295114 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.296016 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.296017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:59.296755 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:59.296758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.297205 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.297207 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:59.297268 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:59.297886 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:59.297898 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:59.300652 (+ 2754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
W20260808 00:42:59.300930 2125 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:59.294410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:58073 (local address 127.20.105.126:33269)
0808 00:42:59.294474 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:59.294477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:59.294526 (+ 49us) server_negotiation.cc:407] Connection header received
0808 00:42:59.294585 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:59.294586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:59.294611 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:59.294682 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:59.295230 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.295901 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.296859 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.297086 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.297155 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:59.300791 (+ 3636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:58073: BlockingRecv error: recv got EOF from 127.20.105.67:58073 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 00:42:59 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:42:59 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:42:59.984247 2128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:42:59.973021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:54911)
0808 00:42:59.973246 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:42:59.973263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 00:42:59.973350 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:42:59.973538 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:42:59.973539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:42:59.973777 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:42:59.973853 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:59.973855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.974741 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.974743 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:59.975327 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:42:59.975329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.975751 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.975752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:42:59.975806 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:59.981820 (+ 6014us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:42:59.981827 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:42:59.984125 (+ 2298us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":118,"threads_started":1}
W20260808 00:42:59.984463 2129 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:42:59.973163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:54911 (local address 127.20.105.126:33269)
0808 00:42:59.973334 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 00:42:59.973336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:42:59.973347 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:42:59.973418 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:42:59.973419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:42:59.973432 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:42:59.973519 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:42:59.973965 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.974628 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.975420 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:42:59.975662 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:42:59.975739 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:42:59.984333 (+ 8594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:54911: BlockingRecv error: recv got EOF from 127.20.105.65:54911 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":89,"threads_started":1}
Aug 08 00:43:00 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:00 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:00.137001 2130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:00.131338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:44655)
0808 00:43:00.131523 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:00.131541 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:00.131622 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:00.131858 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:00.131860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:00.132161 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:00.132234 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:00.132237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.133142 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.133143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:00.133699 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:00.133700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.134107 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.134108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:00.134152 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:00.134776 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:00.134782 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:00.136909 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":117,"threads_started":1}
W20260808 00:43:00.137169 2129 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:00.131435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:44655 (local address 127.20.105.126:33269)
0808 00:43:00.131525 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:00.131528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:00.131672 (+ 144us) server_negotiation.cc:407] Connection header received
0808 00:43:00.131718 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:00.131719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:00.131737 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:00.131791 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:00.132315 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.133063 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.133791 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.134027 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.134098 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:00.137070 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:44655: BlockingRecv error: recv got EOF from 127.20.105.66:44655 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 00:43:00 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:00 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:00.307487 2131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:00.301612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:57765)
0808 00:43:00.301837 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:00.301851 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:00.301930 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:00.302075 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:00.302076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:00.302287 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:00.302359 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:00.302362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.303069 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.303071 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:00.303643 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:00.303646 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.304271 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.304273 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:00.304350 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:00.304922 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:00.304934 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:00.307344 (+ 2410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":148,"threads_started":1}
W20260808 00:43:00.307531 2129 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:00.301682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:57765 (local address 127.20.105.126:33269)
0808 00:43:00.301761 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:00.301764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:00.301922 (+ 158us) server_negotiation.cc:407] Connection header received
0808 00:43:00.301994 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:00.301995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:00.302014 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:00.302068 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:00.302418 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.302982 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.303745 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.303988 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.304133 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:00.307453 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:57765: BlockingRecv error: recv got EOF from 127.20.105.67:57765 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 00:43:00 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:00 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:00.991245 2133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:00.985117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:60817)
0808 00:43:00.985412 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:00.985428 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:00.985525 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:00.985685 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:00.985686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:00.985917 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:00.986015 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:00.986017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.986940 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.986942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:00.987873 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:00.987877 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.988303 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.988304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:00.988349 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:00.989103 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:00.989109 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:00.991165 (+ 2056us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":162,"threads_started":1}
W20260808 00:43:00.991428 2134 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:00.985253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:60817 (local address 127.20.105.126:33269)
0808 00:43:00.985426 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:00.985428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:00.985523 (+ 95us) server_negotiation.cc:407] Connection header received
0808 00:43:00.985588 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:00.985589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:00.985605 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:00.985678 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:00.986100 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.986780 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.987989 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:00.988228 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:00.988296 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:00.991319 (+ 3023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:60817: BlockingRecv error: recv got EOF from 127.20.105.65:60817 (error 108)
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":74,"threads_started":1}
Aug 08 00:43:01 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:01 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:01.143932 2136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:01.137700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:34137)
0808 00:43:01.137976 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:01.137989 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:01.138068 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:01.138279 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:01.138280 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:01.138529 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:01.138603 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:01.138605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.139565 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.139566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:01.140601 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:01.140605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.141114 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.141115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:01.141161 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:01.141723 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:01.141728 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:01.143851 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":151,"threads_started":1}
W20260808 00:43:01.144158 2134 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:01.137788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:34137 (local address 127.20.105.126:33269)
0808 00:43:01.137916 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:01.137920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:01.138120 (+ 200us) server_negotiation.cc:407] Connection header received
0808 00:43:01.138169 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:01.138171 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:01.138189 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:01.138243 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:01.138698 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.139451 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.140709 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.141035 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.141105 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:01.144012 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:34137: BlockingRecv error: recv got EOF from 127.20.105.66:34137 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 08 00:43:01 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:01 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:01.314483 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:01.308250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:38261)
0808 00:43:01.308511 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:01.308524 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:01.308602 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:01.308839 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:01.308847 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:01.309101 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:01.309175 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:01.309177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.310079 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.310081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:01.310714 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:01.310716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.311171 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.311172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:01.311232 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:01.311880 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:01.311886 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:01.314398 (+ 2512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":158,"threads_started":1}
W20260808 00:43:01.314656 2134 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:01.308388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:38261 (local address 127.20.105.126:33269)
0808 00:43:01.308464 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:01.308467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:01.308593 (+ 126us) server_negotiation.cc:407] Connection header received
0808 00:43:01.308658 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:01.308659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:01.308681 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:01.308749 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:01.309289 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.309969 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.310821 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.311062 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.311130 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:01.314512 (+ 3382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:38261: BlockingRecv error: recv got EOF from 127.20.105.67:38261 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 00:43:01 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:01 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:01.997761 2138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:01.991829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:53385)
0808 00:43:01.992071 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:01.992086 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:01.992176 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:01.992499 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:01.992500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:01.992752 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:01.992828 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:01.992830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.993782 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.993784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:01.994462 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:01.994465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.994842 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.994843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:01.994910 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:01.995544 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:01.995552 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:01.997677 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":105,"threads_started":1}
W20260808 00:43:01.997936 2142 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:01.992010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:53385 (local address 127.20.105.126:33269)
0808 00:43:01.992245 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:01.992247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:01.992265 (+ 18us) server_negotiation.cc:407] Connection header received
0808 00:43:01.992300 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:01.992301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:01.992322 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:01.992462 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:01.992936 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.993675 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.994552 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:01.994755 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:01.994809 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:01.997823 (+ 3014us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:53385: BlockingRecv error: recv got EOF from 127.20.105.65:53385 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":109,"threads_started":1}
Aug 08 00:43:02 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:02 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:02.150301 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:02.144607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:33813)
0808 00:43:02.144832 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:02.144848 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:02.144952 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:02.145164 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:02.145166 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:02.145463 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:02.145535 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:02.145538 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:02.146272 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:02.146274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:02.146891 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:02.146894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:02.147348 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:02.147349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:02.147396 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:02.148063 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:02.148069 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:02.150222 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":115,"threads_started":1}
W20260808 00:43:02.150386 2142 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:02.144772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:33813 (local address 127.20.105.126:33269)
0808 00:43:02.144830 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:02.144832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:02.144914 (+ 82us) server_negotiation.cc:407] Connection header received
0808 00:43:02.145053 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:02.145054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:02.145073 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:02.145135 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:02.145611 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:02.146194 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:02.147052 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:02.147278 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:02.147338 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:02.150324 (+ 2986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:33813: BlockingRecv error: recv got EOF from 127.20.105.66:33813 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 08 00:43:02 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:02 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:02.321049 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:02.315121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:55537)
0808 00:43:02.315357 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:02.315369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:02.315479 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:02.315693 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:02.315695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:02.315934 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:02.316001 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:02.316003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:02.316864 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:02.316865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:02.317446 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:02.317447 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:02.317876 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:02.317877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:02.317952 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:02.318592 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:02.318598 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:02.320915 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":126,"threads_started":1}
W20260808 00:43:02.321115 2142 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:02.315195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:55537 (local address 127.20.105.126:33269)
0808 00:43:02.315307 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:02.315309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:02.315450 (+ 141us) server_negotiation.cc:407] Connection header received
0808 00:43:02.315594 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:02.315595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:02.315612 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:02.315682 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:02.316152 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:02.316757 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:02.317540 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:02.317758 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:02.317876 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:02.321030 (+ 3154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:55537: BlockingRecv error: recv got EOF from 127.20.105.67:55537 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 00:43:02 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:02 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:03.004522 2146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:02.998409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:52749)
0808 00:43:02.998652 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:02.998667 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:02.998753 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:02.998980 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:02.998981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:02.999213 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:02.999307 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:02.999309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.000217 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.000218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:03.000849 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:03.000852 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.001321 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.001322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:03.001406 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:03.002033 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:03.002039 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:03.004384 (+ 2345us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":146,"threads_started":1}
W20260808 00:43:03.004554 2148 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:02.998538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:52749 (local address 127.20.105.126:33269)
0808 00:43:02.998724 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:02.998726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:02.998738 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:43:02.998839 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:02.998840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:02.998855 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:02.998932 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:02.999393 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.000063 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.000953 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.001250 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.001320 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:03.004479 (+ 3159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:52749: BlockingRecv error: recv got EOF from 127.20.105.65:52749 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":75,"threads_started":1}
Aug 08 00:43:03 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:03 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:03.157090 2150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:03.150960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:54055)
0808 00:43:03.151204 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:03.151225 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:03.151334 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:03.151594 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:03.151596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:03.151854 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:03.151924 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:03.151926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.152802 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.152803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:03.153448 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:03.153450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.153881 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.153882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:03.153950 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:03.154595 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:03.154606 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:03.156966 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":143,"threads_started":1}
W20260808 00:43:03.157163 2148 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:03.151064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:54055 (local address 127.20.105.126:33269)
0808 00:43:03.151171 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:03.151174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:03.151304 (+ 130us) server_negotiation.cc:407] Connection header received
0808 00:43:03.151478 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:03.151484 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:03.151508 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:03.151585 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:03.152072 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.152696 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.153560 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.153793 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.153884 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:03.157087 (+ 3203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:54055: BlockingRecv error: recv got EOF from 127.20.105.66:54055 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 00:43:03 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:03 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:03.327728 2151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:03.321739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:50475)
0808 00:43:03.321957 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:03.321968 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:03.322045 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:03.322241 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:03.322243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:03.322461 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:03.322535 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:03.322537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.323345 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.323346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:03.323988 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:03.323991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.324492 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.324493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:03.324554 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:03.325199 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:03.325205 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:03.327646 (+ 2441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":130,"threads_started":1}
W20260808 00:43:03.327857 2148 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:03.321871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:50475 (local address 127.20.105.126:33269)
0808 00:43:03.321959 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:03.321962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:03.322043 (+ 81us) server_negotiation.cc:407] Connection header received
0808 00:43:03.322120 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:03.322121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:03.322137 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:03.322187 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:03.322651 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.323246 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.324141 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:03.324380 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:03.324460 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:03.327770 (+ 3310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:50475: BlockingRecv error: recv got EOF from 127.20.105.67:50475 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 00:43:04 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:04 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:04.011152 2152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:04.005186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:36655)
0808 00:43:04.005408 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:04.005421 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:04.005493 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:04.005741 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:04.005742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:04.005981 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:04.006059 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:04.006061 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.006914 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.006915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:04.007509 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:04.007511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.007907 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.007908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:04.007962 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:04.008572 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:04.008578 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:04.011043 (+ 2465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":129,"threads_started":1}
W20260808 00:43:04.011262 2154 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:04.005350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:36655 (local address 127.20.105.126:33269)
0808 00:43:04.005516 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:04.005519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:04.005528 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:43:04.005558 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:04.005559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:04.005595 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:04.005696 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:04.006173 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.006834 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.007614 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.007829 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.007890 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:04.011176 (+ 3286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:36655: BlockingRecv error: recv got EOF from 127.20.105.65:36655 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
Aug 08 00:43:04 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:04 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:04.163933 2155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:04.157748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:47101)
0808 00:43:04.157951 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:04.157964 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:04.158045 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:04.158221 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:04.158223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:04.158511 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:04.158589 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:04.158591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.159666 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.159668 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:04.160418 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:04.160421 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.160819 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.160820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:04.160869 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:04.161459 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:04.161465 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:04.163848 (+ 2383us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":110,"threads_started":1}
W20260808 00:43:04.164124 2154 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:04.157827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:47101 (local address 127.20.105.126:33269)
0808 00:43:04.157932 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:04.157934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:04.158040 (+ 106us) server_negotiation.cc:407] Connection header received
0808 00:43:04.158102 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:04.158102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:04.158116 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:04.158171 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:04.158695 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.159542 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.160514 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.160738 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.160801 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:04.163982 (+ 3181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:47101: BlockingRecv error: recv got EOF from 127.20.105.66:47101 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 08 00:43:04 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:04 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:04.339548 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:04.328413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:60733)
0808 00:43:04.328625 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:04.328638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:04.328716 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:04.328959 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:04.328960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:04.329200 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:04.329270 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:04.329273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.330046 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.330047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:04.330680 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:04.330683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.331137 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.331138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:04.331201 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:04.336819 (+ 5618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:04.336826 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:04.339463 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":147,"threads_started":1}
W20260808 00:43:04.339699 2154 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:04.328517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:60733 (local address 127.20.105.126:33269)
0808 00:43:04.328597 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:04.328600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:04.328716 (+ 116us) server_negotiation.cc:407] Connection header received
0808 00:43:04.328822 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:04.328823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:04.328849 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:04.328926 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:04.329367 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.329954 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.330792 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:04.331024 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:04.331086 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:04.339591 (+ 8505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:60733: BlockingRecv error: recv got EOF from 127.20.105.67:60733 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 00:43:05 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:05 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:05.017699 2158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:05.011815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:60295)
0808 00:43:05.012010 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:05.012063 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:05.012140 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:05.012434 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:05.012436 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:05.012701 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:05.012783 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:05.012785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.013534 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.013535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:05.014140 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:05.014142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.014527 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.014528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:05.014594 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:05.015268 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:05.015276 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:05.017601 (+ 2325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":97,"threads_started":1}
W20260808 00:43:05.017832 2159 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:05.011974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:60295 (local address 127.20.105.126:33269)
0808 00:43:05.012233 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:05.012235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:05.012244 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:43:05.012272 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:05.012272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:05.012285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:05.012360 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:05.012874 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.013436 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.014223 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.014429 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.014486 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:05.017730 (+ 3244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:60295: BlockingRecv error: recv got EOF from 127.20.105.65:60295 (error 108)
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":176,"threads_started":1}
Aug 08 00:43:05 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:05 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:05.171379 2160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:05.164778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:35457)
0808 00:43:05.165008 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:05.165023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:05.165169 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:05.165435 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:05.165436 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:05.165697 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:05.165780 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:05.165783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.166962 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.166963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:05.167639 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:05.167642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.168160 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.168161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:05.168244 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:05.168825 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:05.168831 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:05.171293 (+ 2462us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":155,"threads_started":1}
W20260808 00:43:05.171599 2159 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:05.164877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:35457 (local address 127.20.105.126:33269)
0808 00:43:05.165002 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:05.165005 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:05.165148 (+ 143us) server_negotiation.cc:407] Connection header received
0808 00:43:05.165288 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:05.165290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:05.165308 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:05.165408 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:05.165988 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.166836 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.167761 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.168011 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.168150 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:05.171528 (+ 3378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:35457: BlockingRecv error: recv got EOF from 127.20.105.66:35457 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 00:43:05 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:05 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:05.346616 2162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:05.340350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:56111)
0808 00:43:05.340627 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:05.340639 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:05.340746 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:05.340973 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:05.340974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:05.341210 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:05.341279 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:05.341282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.342113 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.342115 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:05.342698 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:05.342700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.343202 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.343204 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:05.343301 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:05.343871 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:05.343877 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:05.346536 (+ 2659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":162,"threads_started":1}
W20260808 00:43:05.346817 2159 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:05.340479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:56111 (local address 127.20.105.126:33269)
0808 00:43:05.340573 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:05.340576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:05.340719 (+ 143us) server_negotiation.cc:407] Connection header received
0808 00:43:05.340827 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:05.340828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:05.340854 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:05.340963 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:05.341393 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.342005 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.342796 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:05.343011 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:05.343187 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:05.346728 (+ 3541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:56111: BlockingRecv error: recv got EOF from 127.20.105.67:56111 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 00:43:06 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:06 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:06.024338 2163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:06.018444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:60623)
0808 00:43:06.018667 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:06.018681 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:06.018758 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:06.018957 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:06.018958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:06.019207 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:06.019280 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:06.019282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.020112 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.020113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:06.020682 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:06.020684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.021265 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.021266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:06.021351 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:06.022005 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:06.022010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:06.024256 (+ 2246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":124,"threads_started":1}
W20260808 00:43:06.024580 2164 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:06.018591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:60623 (local address 127.20.105.126:33269)
0808 00:43:06.018762 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:06.018764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:06.018796 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:43:06.018828 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:06.018829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:06.018843 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:06.018929 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:06.019371 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.019958 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.020787 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.021159 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.021238 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:06.024455 (+ 3217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:60623: BlockingRecv error: recv got EOF from 127.20.105.65:60623 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":89,"threads_started":1}
Aug 08 00:43:06 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:06 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:06.180063 2165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:06.172095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:36129)
0808 00:43:06.172302 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:06.172321 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:06.172426 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:06.172672 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:06.172673 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:06.172986 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:06.173076 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:06.173081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.174379 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.174381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:06.175409 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:06.175413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.175995 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.175997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:06.176108 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:06.176882 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:06.176888 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:06.179897 (+ 3009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":94,"threads_started":1}
W20260808 00:43:06.180280 2164 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:06.172171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:36129 (local address 127.20.105.126:33269)
0808 00:43:06.172274 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:06.172277 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:06.172400 (+ 123us) server_negotiation.cc:407] Connection header received
0808 00:43:06.172537 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:06.172539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:06.172556 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:06.172643 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:06.173274 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.174286 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.175509 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.175868 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.175980 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:06.180113 (+ 4133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:36129: BlockingRecv error: recv got EOF from 127.20.105.66:36129 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 00:43:06 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:06 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:06.353462 2167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:06.347345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:60983)
0808 00:43:06.347548 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:06.347562 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:06.347656 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:06.347856 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:06.347857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:06.348118 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:06.348189 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:06.348192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.349008 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.349009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:06.349633 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:06.349636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.350233 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.350234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:06.350296 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:06.350939 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:06.350945 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:06.353334 (+ 2389us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":115,"threads_started":1}
W20260808 00:43:06.353520 2164 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:06.347415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:60983 (local address 127.20.105.126:33269)
0808 00:43:06.347493 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:06.347496 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:06.347633 (+ 137us) server_negotiation.cc:407] Connection header received
0808 00:43:06.347755 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:06.347756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:06.347781 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:06.347828 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:06.348265 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.348851 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.349895 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:06.350111 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:06.350189 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:06.353440 (+ 3251us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:60983: BlockingRecv error: recv got EOF from 127.20.105.67:60983 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 00:43:07 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:07 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:07.030777 2168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:07.024927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:50019)
0808 00:43:07.025143 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:07.025160 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:07.025234 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:07.025391 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:07.025391 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:07.025609 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:07.025678 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:07.025681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.026446 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.026447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:07.027034 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:07.027035 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.027463 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.027464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:07.027519 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:07.028203 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:07.028209 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:07.030687 (+ 2478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":129,"threads_started":1}
W20260808 00:43:07.030992 2169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:07.025026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:50019 (local address 127.20.105.126:33269)
0808 00:43:07.025176 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:07.025178 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:07.025236 (+ 58us) server_negotiation.cc:407] Connection header received
0808 00:43:07.025285 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:07.025286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:07.025301 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:07.025374 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:07.025784 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.026363 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.027133 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.027361 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.027446 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:07.030872 (+ 3426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:50019: BlockingRecv error: recv got EOF from 127.20.105.65:50019 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
Aug 08 00:43:07 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:07 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:07.187084 2170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:07.180874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:38345)
0808 00:43:07.181138 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:07.181154 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:07.181257 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:07.181449 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:07.181450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:07.181678 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:07.181749 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:07.181751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.182568 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.182569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:07.183157 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:07.183160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.183617 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.183619 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:07.183682 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:07.184487 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:07.184493 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:07.186977 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":159,"threads_started":1}
W20260808 00:43:07.187263 2169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:07.180928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:38345 (local address 127.20.105.126:33269)
0808 00:43:07.181013 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:07.181016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:07.181249 (+ 233us) server_negotiation.cc:407] Connection header received
0808 00:43:07.181326 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:07.181327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:07.181355 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:07.181425 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:07.181851 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.182486 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.183262 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.183494 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.183568 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:07.187127 (+ 3559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:38345: BlockingRecv error: recv got EOF from 127.20.105.66:38345 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 00:43:07 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:07 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:07.360445 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:07.354188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:41199)
0808 00:43:07.354380 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:07.354415 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:07.354503 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:07.354720 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:07.354721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:07.354984 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:07.355055 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:07.355057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.355888 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.355890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:07.356632 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:07.356634 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.357074 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.357075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:07.357151 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:07.357765 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:07.357771 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:07.360335 (+ 2564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":104,"threads_started":1}
W20260808 00:43:07.360678 2169 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:07.354356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:41199 (local address 127.20.105.126:33269)
0808 00:43:07.354450 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:07.354453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:07.354474 (+ 21us) server_negotiation.cc:407] Connection header received
0808 00:43:07.354599 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:07.354601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:07.354628 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:07.354706 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:07.355149 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.355783 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.356745 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:07.356982 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:07.357074 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:07.360472 (+ 3398us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:41199: BlockingRecv error: recv got EOF from 127.20.105.67:41199 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 00:43:08 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:08 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:08.037317 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:08.031385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:59033)
0808 00:43:08.031584 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:08.031605 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:08.031681 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:08.032100 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:08.032101 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:08.032330 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:08.032401 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:08.032402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.033222 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.033224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:08.033802 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:08.033804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.034264 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.034265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:08.034325 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:08.035004 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:08.035010 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:08.037218 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":114,"threads_started":1}
W20260808 00:43:08.037495 2176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:08.031571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:59033 (local address 127.20.105.126:33269)
0808 00:43:08.031872 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:08.031876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:08.031896 (+ 20us) server_negotiation.cc:407] Connection header received
0808 00:43:08.031932 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:08.031933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:08.031949 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:08.032061 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:08.032500 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.033107 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.033916 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.034167 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.034243 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:08.037382 (+ 3139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:59033: BlockingRecv error: recv got EOF from 127.20.105.65:59033 (error 108)
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":229,"threads_started":1}
Aug 08 00:43:08 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:08 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:08.193861 2177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:08.187730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:58501)
0808 00:43:08.187980 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:08.187994 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:08.188128 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:08.188292 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:08.188293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:08.188558 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:08.188625 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:08.188628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.189407 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.189409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:08.189981 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:08.189982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.190409 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.190410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:08.190497 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:08.191067 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:08.191073 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:08.193719 (+ 2646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":124,"threads_started":1}
W20260808 00:43:08.193923 2176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:08.187824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:58501 (local address 127.20.105.126:33269)
0808 00:43:08.187975 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:08.187978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:08.188103 (+ 125us) server_negotiation.cc:407] Connection header received
0808 00:43:08.188213 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:08.188215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:08.188230 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:08.188287 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:08.188716 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.189303 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.190070 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.190313 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.190406 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:08.193836 (+ 3430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:58501: BlockingRecv error: recv got EOF from 127.20.105.66:58501 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 08 00:43:08 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:08 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:08.367123 2178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:08.361194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:52689)
0808 00:43:08.361448 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:08.361462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:08.361580 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:08.361772 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:08.361773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:08.362051 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:08.362116 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:08.362119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.362907 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.362908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:08.363492 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:08.363494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.363871 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.363872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:08.363933 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:08.364658 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:08.364663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:08.367027 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":147,"threads_started":1}
W20260808 00:43:08.367301 2176 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:08.361326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:52689 (local address 127.20.105.126:33269)
0808 00:43:08.361421 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:08.361424 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:08.361551 (+ 127us) server_negotiation.cc:407] Connection header received
0808 00:43:08.361658 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:08.361660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:08.361687 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:08.361762 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:08.362212 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.362804 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.363582 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:08.363792 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:08.363874 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:08.367192 (+ 3318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:52689: BlockingRecv error: recv got EOF from 127.20.105.67:52689 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 00:43:09 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:09 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:09.043697 2181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:09.037986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:36719)
0808 00:43:09.038184 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:09.038196 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:09.038257 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:09.038527 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:09.038528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:09.038766 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:09.038836 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:09.038839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.039728 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.039729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:09.040376 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:09.040380 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.040782 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.040783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:09.040838 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:09.041483 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:09.041488 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:09.043610 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":86,"threads_started":1}
W20260808 00:43:09.043979 2183 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:09.038166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:36719 (local address 127.20.105.126:33269)
0808 00:43:09.038322 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:09.038324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:09.038334 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:43:09.038366 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:09.038367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:09.038381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:09.038468 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:09.038956 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.039598 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.040465 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.040700 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.040766 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:09.043842 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:36719: BlockingRecv error: recv got EOF from 127.20.105.65:36719 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":87,"threads_started":1}
Aug 08 00:43:09 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:09 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:09.200305 2184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:09.194493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:41301)
0808 00:43:09.194697 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:09.194711 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:09.194791 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:09.194992 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:09.194993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:09.195237 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:09.195310 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:09.195312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.196172 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.196173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:09.196775 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:09.196777 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.197215 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.197216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:09.197272 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:09.197837 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:09.197842 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:09.200180 (+ 2338us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":108,"threads_started":1}
W20260808 00:43:09.200368 2183 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:09.194631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:41301 (local address 127.20.105.126:33269)
0808 00:43:09.194724 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:09.194726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:09.194766 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:43:09.194886 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:09.194887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:09.194903 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:09.194962 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:09.195408 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.196058 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.196893 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.197109 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.197180 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:09.200291 (+ 3111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:41301: BlockingRecv error: recv got EOF from 127.20.105.66:41301 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 00:43:09 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:09 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:09.379522 2185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:09.367921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:51593)
0808 00:43:09.368220 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:09.368231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:09.368353 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:09.368558 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:09.368559 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:09.368887 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:09.368969 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:09.368972 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.369760 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.369762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:09.370347 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:09.370349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.370757 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.370759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:09.370821 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:09.376994 (+ 6173us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:09.377002 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:09.379407 (+ 2405us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":184,"threads_started":1}
W20260808 00:43:09.379729 2183 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:09.368068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:51593 (local address 127.20.105.126:33269)
0808 00:43:09.368163 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:09.368166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:09.368300 (+ 134us) server_negotiation.cc:407] Connection header received
0808 00:43:09.368436 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:09.368437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:09.368476 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:09.368537 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:09.369072 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.369653 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.370437 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:09.370657 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:09.370721 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:09.379647 (+ 8926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:51593: BlockingRecv error: recv got EOF from 127.20.105.67:51593 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 00:43:10 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:10 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:10.050395 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:10.044410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:48353)
0808 00:43:10.044588 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:10.044607 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:10.044679 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:10.044864 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:10.044865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:10.045113 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:10.045188 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:10.045191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.046092 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.046094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:10.046684 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:10.046687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.047176 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.047177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:10.047241 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:10.047893 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:10.047903 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:10.050263 (+ 2360us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":117,"threads_started":1}
W20260808 00:43:10.050472 2188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:10.044525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:48353 (local address 127.20.105.126:33269)
0808 00:43:10.044696 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:10.044699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:10.044710 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:43:10.044747 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:10.044748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:10.044763 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:10.044846 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:10.045265 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.045995 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.046799 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.047062 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.047156 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:10.050383 (+ 3227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:48353: BlockingRecv error: recv got EOF from 127.20.105.65:48353 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
Aug 08 00:43:10 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:10 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:10.207535 2190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:10.201069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:42501)
0808 00:43:10.201278 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:10.201297 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:10.201367 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:10.201696 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:10.201698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:10.201933 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:10.202003 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:10.202005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.203069 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.203070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:10.203821 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:10.203824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.204253 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.204254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:10.204307 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:10.204919 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:10.204925 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:10.207415 (+ 2490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":125,"threads_started":1}
W20260808 00:43:10.207860 2188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:10.201353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:42501 (local address 127.20.105.126:33269)
0808 00:43:10.201444 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:10.201446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:10.201457 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:43:10.201499 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:10.201500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:10.201516 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:10.201603 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:10.202136 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.202940 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.203910 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.204157 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.204225 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:10.207783 (+ 3558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:42501: BlockingRecv error: recv got EOF from 127.20.105.66:42501 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 00:43:10 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:10 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:10.386301 2192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:10.380398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:36099)
0808 00:43:10.380646 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:10.380660 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:10.380751 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:10.380956 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:10.380958 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:10.381198 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:10.381269 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:10.381271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.382094 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.382096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:10.383018 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:10.383020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.383373 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.383374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:10.383423 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:10.383976 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:10.383982 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:10.386201 (+ 2219us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":148,"threads_started":1}
W20260808 00:43:10.386389 2188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:10.380471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:36099 (local address 127.20.105.126:33269)
0808 00:43:10.380560 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:10.380563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:10.380743 (+ 180us) server_negotiation.cc:407] Connection header received
0808 00:43:10.380815 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:10.380816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:10.380835 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:10.380894 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:10.381351 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.381961 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.383103 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:10.383298 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:10.383358 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:10.386327 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:36099: BlockingRecv error: recv got EOF from 127.20.105.67:36099 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 08 00:43:11 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:11 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:11.056761 2193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:11.051185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:49431)
0808 00:43:11.051402 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:11.051415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:11.051487 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:11.051649 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:11.051650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:11.051882 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:11.051953 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:11.051955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.052781 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.052782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:11.053435 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:11.053438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.053828 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.053829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:11.053874 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:11.054496 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:11.054501 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:11.056665 (+ 2164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":133,"threads_started":1}
W20260808 00:43:11.056945 2194 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:11.051311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:49431 (local address 127.20.105.126:33269)
0808 00:43:11.051488 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:11.051490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:11.051521 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:43:11.051552 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:11.051553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:11.051566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:11.051633 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:11.052077 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.052669 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.053529 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.053755 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.053815 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:11.056787 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:49431: BlockingRecv error: recv got EOF from 127.20.105.65:49431 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
Aug 08 00:43:11 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:11 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:11.214164 2196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:11.208271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:58513)
0808 00:43:11.208462 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:11.208473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:11.208546 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:11.208700 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:11.208701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:11.208983 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:11.209063 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:11.209066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.209916 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.209918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:11.210583 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:11.210586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.211031 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.211033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:11.211094 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:11.211680 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:11.211687 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:11.214080 (+ 2393us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
W20260808 00:43:11.214337 2194 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:11.208381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:58513 (local address 127.20.105.126:33269)
0808 00:43:11.208462 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:11.208464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:11.208524 (+ 60us) server_negotiation.cc:407] Connection header received
0808 00:43:11.208604 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:11.208605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:11.208620 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:11.208689 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:11.209155 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.209768 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.210688 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.210919 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.210992 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:11.214234 (+ 3242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:58513: BlockingRecv error: recv got EOF from 127.20.105.66:58513 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 00:43:11 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:11 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.67@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:11.392573 2198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:11.387035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.67:50779)
0808 00:43:11.387236 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:11.387249 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:11.387323 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:11.387474 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:11.387474 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:11.387718 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:11.387789 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:11.387791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.388582 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.388583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:11.389190 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:11.389193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.389584 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.389584 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:11.389631 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:11.390207 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:11.390213 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:11.392446 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":119,"threads_started":1}
W20260808 00:43:11.392606 2194 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:11.387142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:50779 (local address 127.20.105.126:33269)
0808 00:43:11.387241 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:11.387244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:11.387319 (+ 75us) server_negotiation.cc:407] Connection header received
0808 00:43:11.387374 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:11.387375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:11.387390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:11.387452 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:11.387874 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.388489 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.389288 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:11.389503 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:11.389574 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:11.392540 (+ 2966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.67:50779: BlockingRecv error: recv got EOF from 127.20.105.67:50779 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 00:43:12 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:12 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.65@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:12.063709 2199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:12.057461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.65:52685)
0808 00:43:12.057643 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:12.057664 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:12.057744 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:12.057963 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:12.057965 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:12.058203 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:12.058281 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:12.058283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.059168 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.059170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:12.059799 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:12.059801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.060229 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.060230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:12.060286 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.061190 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:12.061197 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:12.063621 (+ 2424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
W20260808 00:43:12.063952 2200 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:12.057609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:52685 (local address 127.20.105.126:33269)
0808 00:43:12.057763 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:12.057765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:12.057775 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:43:12.057810 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:12.057811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:12.057823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:12.057901 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:12.058423 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.059027 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.059895 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.060147 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.060211 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.063754 (+ 3543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.65:52685: BlockingRecv error: recv got EOF from 127.20.105.65:52685 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
Aug 08 00:43:12 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:43:12 dist-test-slave-kvfs krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.105.66@KRBTEST.COM for oryx/127.20.105.126@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:43:12.220969 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:12.214996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33269 (local address 127.20.105.66:55801)
0808 00:43:12.215210 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:12.215233 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:12.215320 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:12.215526 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:12.215528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:12.215748 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:12.215814 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:12.215816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.216827 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.216829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:12.217627 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:12.217630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.218000 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.218001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:12.218048 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.218602 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:12.218608 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:12.220835 (+ 2227us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.105.126:33269: Server oryx/127.20.105.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
W20260808 00:43:12.221028 2200 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:12.215161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:55801 (local address 127.20.105.126:33269)
0808 00:43:12.215221 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:12.215223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:12.215321 (+ 98us) server_negotiation.cc:407] Connection header received
0808 00:43:12.215404 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:12.215405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:12.215420 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:12.215474 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:12.215951 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.216655 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.217714 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.217930 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.217996 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.220951 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.105.66:55801: BlockingRecv error: recv got EOF from 127.20.105.66:55801 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:43:12.282205 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 1510
I20260808 00:43:12.296509 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 1731
I20260808 00:43:12.311570 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 1921
I20260808 00:43:12.323138 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 1402
2026-08-08T00:43:12Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18142 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T00:43:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:43:12Z Disabled control of system clock
I20260808 00:43:12.363781 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44433
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:44433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 00:43:12.459941 2205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:12.460294 2205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:12.460317 2205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:12.461580 2205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:43:12.461625 2205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:12.461638 2205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:12.461651 2205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:43:12.461663 2205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:43:12.463055 2205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:44433
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:12.463310 2205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:12.463708 2205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:12.470697 2216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:12.470785 2215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:12.470763 2205 server_base.cc:1034] running on GCE node
W20260808 00:43:12.470831 2220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:12.471274 2205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:12.471634 2205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:12.472831 2205 hybrid_clock.cc:648] HybridClock initialized: now 1786149792472802 us; error 46 us; skew 500 ppm
I20260808 00:43:12.474462 2205 webserver.cc:460] Webserver started at http://127.20.105.126:42657/ using document root <none> and password file <none>
I20260808 00:43:12.474777 2205 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:12.474833 2205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:12.475066 2205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:12.476542 2205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "5ed22be906cb456187ff1252d61fb7bb"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "158612bdb06c1c16384180b61d7b6d71"
server_key_iv: "8bef3b6eeab7dd38e9c51c95dc4c0a2e"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.477008 2205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "5ed22be906cb456187ff1252d61fb7bb"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "158612bdb06c1c16384180b61d7b6d71"
server_key_iv: "8bef3b6eeab7dd38e9c51c95dc4c0a2e"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.478629 2205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:43:12.479660 2227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.479918 2205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:12.480079 2205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "5ed22be906cb456187ff1252d61fb7bb"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "158612bdb06c1c16384180b61d7b6d71"
server_key_iv: "8bef3b6eeab7dd38e9c51c95dc4c0a2e"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.480178 2205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:12.508737 2205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:12.509227 2205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:12.509415 2205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:12.514456 2205 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:44433
I20260808 00:43:12.514508 2305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:44433 every 8 connection(s)
I20260808 00:43:12.514963 2205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:43:12.515828 2307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:43:12.519110 2307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb: Bootstrap starting.
I20260808 00:43:12.519853 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 2205
I20260808 00:43:12.519969 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:43:12.520004 2307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb: Neither blocks nor log segments found. Creating new log.
I20260808 00:43:12.520282 20901 external_mini_cluster.cc:1428] Setting key 3fac38979a46363c126baa9c3751475b
I20260808 00:43:12.520411 2307 log.cc:824] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb: Log is configured to *not* fsync() on all Append() calls
I20260808 00:43:12.521524 2307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb: No bootstrap required, opened a new log
I20260808 00:43:12.524478 2307 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed22be906cb456187ff1252d61fb7bb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44433 } }
I20260808 00:43:12.524649 2307 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:43:12.524674 2307 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ed22be906cb456187ff1252d61fb7bb, State: Initialized, Role: FOLLOWER
I20260808 00:43:12.524832 2307 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed22be906cb456187ff1252d61fb7bb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44433 } }
I20260808 00:43:12.524931 2307 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:43:12.524978 2307 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:43:12.525035 2307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:43:12.525846 2307 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed22be906cb456187ff1252d61fb7bb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44433 } }
I20260808 00:43:12.525975 2307 leader_election.cc:302] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ed22be906cb456187ff1252d61fb7bb; no voters:
I20260808 00:43:12.526158 2307 leader_election.cc:288] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:43:12.526332 2313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:43:12.526538 2307 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:43:12.526607 2313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [term 1 LEADER]: Becoming Leader. State: Replica: 5ed22be906cb456187ff1252d61fb7bb, State: Running, Role: LEADER
I20260808 00:43:12.526746 2313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed22be906cb456187ff1252d61fb7bb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44433 } }
I20260808 00:43:12.527164 2313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ed22be906cb456187ff1252d61fb7bb. Latest consensus state: current_term: 1 leader_uuid: "5ed22be906cb456187ff1252d61fb7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed22be906cb456187ff1252d61fb7bb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44433 } } }
I20260808 00:43:12.527267 2313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [sys.catalog]: This master's current role is: LEADER
I20260808 00:43:12.527559 2317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:43:12.527643 2314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ed22be906cb456187ff1252d61fb7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ed22be906cb456187ff1252d61fb7bb" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44433 } } }
I20260808 00:43:12.527772 2314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb [sys.catalog]: This master's current role is: LEADER
I20260808 00:43:12.528475 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:12.520956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57792 (local address 127.20.105.126:44433)
0808 00:43:12.521192 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:12.521203 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:12.521249 (+ 46us) server_negotiation.cc:407] Connection header received
0808 00:43:12.521966 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:12.521977 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:12.522077 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:12.522274 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:12.522497 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.524091 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.525170 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.525823 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.525973 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.526943 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:12.526948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:12.526948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:43:12.527207 (+ 259us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:12.527286 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:43:12.527305 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:12.528093 (+ 788us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:12.528131 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 00:43:12.528224 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260808 00:43:12.529397 2317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:43:12.531368 2317 catalog_manager.cc:1347] Generated new cluster ID: 0fe241eef7cd4bc4aeaef586c5830829
I20260808 00:43:12.531435 2317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:43:12.546957 2317 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:43:12.547825 2317 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:43:12.555130 2317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ed22be906cb456187ff1252d61fb7bb: Generated new TSK 0
I20260808 00:43:12.555420 2317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:43:12.558081 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44433
--builtin_ntp_servers=127.20.105.84:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 00:43:12.658747 2347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:12.658998 2347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:12.659020 2347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:12.660337 2347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:12.660378 2347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:12.660410 2347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:43:12.661840 2347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:12.662144 2347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:12.662508 2347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:12.669121 2355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:12.669133 2352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:12.669134 2353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:12.669691 2347 server_base.cc:1034] running on GCE node
I20260808 00:43:12.669952 2347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:12.670251 2347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:12.671435 2347 hybrid_clock.cc:648] HybridClock initialized: now 1786149792671418 us; error 34 us; skew 500 ppm
I20260808 00:43:12.673060 2347 webserver.cc:460] Webserver started at http://127.20.105.65:37879/ using document root <none> and password file <none>
I20260808 00:43:12.673360 2347 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:12.673415 2347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:12.673568 2347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:12.674814 2347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "0562dbe8f94440f191eaa66ec3954a47"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "7a3530376b96002bb10d1558edb6fd6c"
server_key_iv: "3acc899fa8d10d8ca3970658532cbcea"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.675235 2347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "0562dbe8f94440f191eaa66ec3954a47"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "7a3530376b96002bb10d1558edb6fd6c"
server_key_iv: "3acc899fa8d10d8ca3970658532cbcea"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.676893 2347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 00:43:12.677793 2365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.678025 2347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:12.678159 2347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "0562dbe8f94440f191eaa66ec3954a47"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "7a3530376b96002bb10d1558edb6fd6c"
server_key_iv: "3acc899fa8d10d8ca3970658532cbcea"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.678251 2347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:12.695837 2347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:12.696363 2347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:12.696545 2347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:12.696823 2347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:12.697252 2347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:12.697320 2347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.697371 2347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:12.697393 2347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.704779 2347 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:43485
I20260808 00:43:12.704866 2526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:43485 every 8 connection(s)
I20260808 00:43:12.705459 2347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:43:12.708819 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 2347
I20260808 00:43:12.708910 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:43:12.709151 20901 external_mini_cluster.cc:1428] Setting key 501f1a1d41bc2a019b273f72c79cd746
I20260808 00:43:12.710335 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44433
--builtin_ntp_servers=127.20.105.84:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 00:43:12.712615 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:12.707026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:48059 (local address 127.20.105.126:44433)
0808 00:43:12.707119 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:12.707122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:12.707136 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:43:12.707741 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:12.707742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:12.707758 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:12.707811 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:12.708728 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.709443 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.710813 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.711123 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.711210 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.712249 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:12.712251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:12.712252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:43:12.712258 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:12.712283 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:43:12.712288 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:12.712351 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:12.712440 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 00:43:12.712474 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 00:43:12.712772 2529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:12.706566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44433 (local address 127.20.105.65:48059)
0808 00:43:12.706814 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:12.706866 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:12.707598 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:12.707900 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:12.707905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:12.707918 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:12.708623 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:12.708629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.709587 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.709588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:12.710723 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:12.710726 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.711221 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.711222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:12.711351 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.712054 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:43:12.712055 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:12.712118 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:43:12.712155 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:12.712356 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:12.712362 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:12.712365 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:43:12.712424 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:43:12.712535 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260808 00:43:12.713460 2527 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44433
I20260808 00:43:12.713599 2527 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:12.713814 2527 heartbeater.cc:511] Master 127.20.105.126:44433 requested a full tablet report, sending...
I20260808 00:43:12.714458 2254 ts_manager.cc:194] Registered new tserver with Master: 0562dbe8f94440f191eaa66ec3954a47 (127.20.105.65:43485)
I20260808 00:43:12.715154 2254 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.65:48059
W20260808 00:43:12.811092 2530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:12.811434 2530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:12.811476 2530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:12.812817 2530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:12.812902 2530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:12.812959 2530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:43:12.814503 2530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:12.814850 2530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:12.815243 2530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:12.821413 2537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:12.821384 2540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:12.821393 2538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:12.821755 2530 server_base.cc:1034] running on GCE node
I20260808 00:43:12.822038 2530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:12.822336 2530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:12.823567 2530 hybrid_clock.cc:648] HybridClock initialized: now 1786149792823544 us; error 37 us; skew 500 ppm
I20260808 00:43:12.825285 2530 webserver.cc:460] Webserver started at http://127.20.105.66:35077/ using document root <none> and password file <none>
I20260808 00:43:12.825598 2530 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:12.825688 2530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:12.825861 2530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:12.827399 2530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "ca97b6cfdeaf41ef9ba894a13ef0db0b"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "fe95efe93103018a8268abf2c8e08c4c"
server_key_iv: "256cd229ac6b12ad3f46364e19b04101"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.827849 2530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "ca97b6cfdeaf41ef9ba894a13ef0db0b"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "fe95efe93103018a8268abf2c8e08c4c"
server_key_iv: "256cd229ac6b12ad3f46364e19b04101"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.829612 2530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:43:12.830569 2552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.830780 2530 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:43:12.830901 2530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "ca97b6cfdeaf41ef9ba894a13ef0db0b"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "fe95efe93103018a8268abf2c8e08c4c"
server_key_iv: "256cd229ac6b12ad3f46364e19b04101"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.831009 2530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:12.842516 2530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:12.842967 2530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:12.843164 2530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:12.843459 2530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:12.843902 2530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:12.843971 2530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.844077 2530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:12.844110 2530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.851392 2530 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:40593
I20260808 00:43:12.851445 2718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:40593 every 8 connection(s)
I20260808 00:43:12.851873 2530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:43:12.858490 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:12.852851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:36359 (local address 127.20.105.126:44433)
0808 00:43:12.852971 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:12.852974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:12.853218 (+ 244us) server_negotiation.cc:407] Connection header received
0808 00:43:12.853856 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:12.853857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:12.853873 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:12.853949 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:12.854794 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.855547 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.856718 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.856950 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.857031 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.858116 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:12.858117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:12.858118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:43:12.858124 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:12.858144 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:43:12.858148 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:12.858213 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:12.858360 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 00:43:12.858396 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 00:43:12.858724 2724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:12.852819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44433 (local address 127.20.105.66:36359)
0808 00:43:12.853056 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:12.853114 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:12.853720 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:12.854062 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:12.854066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:12.854077 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:12.854705 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:12.854709 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.855647 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.855648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:12.856594 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:12.856596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:12.857056 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:12.857057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:12.857183 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:12.857898 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:43:12.857899 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:12.857936 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:43:12.857965 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:12.858248 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:12.858255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:12.858259 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:43:12.858338 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:43:12.858448 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260808 00:43:12.859236 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 2530
I20260808 00:43:12.859344 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:43:12.859406 2719 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44433
I20260808 00:43:12.859565 2719 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:12.859618 20901 external_mini_cluster.cc:1428] Setting key d4bfc5c31b292ba0a84281d8e2caa666
I20260808 00:43:12.859804 2719 heartbeater.cc:511] Master 127.20.105.126:44433 requested a full tablet report, sending...
I20260808 00:43:12.860270 2253 ts_manager.cc:194] Registered new tserver with Master: ca97b6cfdeaf41ef9ba894a13ef0db0b (127.20.105.66:40593)
I20260808 00:43:12.860824 2253 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.66:36359
I20260808 00:43:12.860975 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44433
--builtin_ntp_servers=127.20.105.84:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 00:43:12.957733 2725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:12.958017 2725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:12.958042 2725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:12.959280 2725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:12.959336 2725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:12.959364 2725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:43:12.960897 2725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:12.961225 2725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:12.961599 2725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:12.967407 2734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:12.967559 2731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:12.967582 2732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:12.968051 2725 server_base.cc:1034] running on GCE node
I20260808 00:43:12.968341 2725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:12.968703 2725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:12.969887 2725 hybrid_clock.cc:648] HybridClock initialized: now 1786149792969868 us; error 51 us; skew 500 ppm
I20260808 00:43:12.971546 2725 webserver.cc:460] Webserver started at http://127.20.105.67:33145/ using document root <none> and password file <none>
I20260808 00:43:12.971843 2725 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:12.971920 2725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:12.972159 2725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:12.973531 2725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "86bfe0bb8ff942fcba8d3f4c0acc7f96"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "fbe310c46de248e3fbd53a1e17323345"
server_key_iv: "86b611e9239de13c521d0879e0610a6a"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.973980 2725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "86bfe0bb8ff942fcba8d3f4c0acc7f96"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "fbe310c46de248e3fbd53a1e17323345"
server_key_iv: "86b611e9239de13c521d0879e0610a6a"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.975534 2725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:43:12.976555 2747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.976783 2725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:43:12.976910 2725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "86bfe0bb8ff942fcba8d3f4c0acc7f96"
format_stamp: "Formatted at 2026-08-08 00:43:12 on dist-test-slave-kvfs"
server_key: "fbe310c46de248e3fbd53a1e17323345"
server_key_iv: "86b611e9239de13c521d0879e0610a6a"
server_key_version: "encryptionkey@0"
I20260808 00:43:12.977001 2725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:12.995404 2725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:12.995852 2725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:12.996081 2725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:12.996415 2725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:12.996868 2725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:12.996932 2725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:12.996979 2725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:12.997009 2725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.003773 2725 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:34711
I20260808 00:43:13.003916 2902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:34711 every 8 connection(s)
I20260808 00:43:13.004334 2725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:43:13.009024 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 2725
I20260808 00:43:13.009262 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:43:13.009516 20901 external_mini_cluster.cc:1428] Setting key d1c93aee47c862c9d1ff10343d18196f
I20260808 00:43:13.010967 2311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.005335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:33333 (local address 127.20.105.126:44433)
0808 00:43:13.005415 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:13.005417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:13.005691 (+ 274us) server_negotiation.cc:407] Connection header received
0808 00:43:13.006374 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:13.006376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:13.006393 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:13.006465 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:13.007363 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:13.007950 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:13.009099 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:13.009371 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:13.009458 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:13.010657 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:13.010658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:13.010659 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:43:13.010666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:13.010690 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:43:13.010695 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:13.010753 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:13.010855 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 00:43:13.010883 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:43:13.011268 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.005290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44433 (local address 127.20.105.67:33333)
0808 00:43:13.005533 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:13.005576 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:13.006240 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:13.006559 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:13.006563 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:13.006573 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:13.007243 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:13.007249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:13.008106 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:13.008107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:13.008992 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:13.008995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:13.009467 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:13.009469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:13.009683 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:13.010455 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:43:13.010456 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:13.010495 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:43:13.010566 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:13.010769 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:13.010774 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:13.010778 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:43:13.010841 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:43:13.010954 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":93,"threads_started":1}
I20260808 00:43:13.011873 2903 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44433
I20260808 00:43:13.012009 2903 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:13.012285 2903 heartbeater.cc:511] Master 127.20.105.126:44433 requested a full tablet report, sending...
I20260808 00:43:13.012857 2253 ts_manager.cc:194] Registered new tserver with Master: 86bfe0bb8ff942fcba8d3f4c0acc7f96 (127.20.105.67:34711)
I20260808 00:43:13.013337 2253 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.67:33333
I20260808 00:43:13.021879 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:43:13.024339 2311 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:13.023800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57806 (local address 127.20.105.126:44433)
0808 00:43:13.023878 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:13.023881 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:13.023892 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:43:13.023953 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:13.023954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:13.023966 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:13.024040 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:13.024228 (+ 188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57806: BlockingRecv error: recv got EOF from 127.0.0.1:57806 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:43:13.024367 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 2347
I20260808 00:43:13.043313 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 2530
I20260808 00:43:13.055294 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 2725
I20260808 00:43:13.080104 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 2205
2026-08-08T00:43:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (758 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T00:43:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:43:13Z Disabled control of system clock
I20260808 00:43:13.123378 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33063
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:40179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:33063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:43:13.221526 2918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:13.221812 2918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:13.221837 2918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:13.223044 2918 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:43:13.223079 2918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:13.223093 2918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:13.223106 2918 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:43:13.223119 2918 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:43:13.224607 2918 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:33063
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:13.224879 2918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:13.225196 2918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:13.231374 2923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:13.231455 2925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:13.231382 2928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:13.231736 2918 server_base.cc:1034] running on GCE node
I20260808 00:43:13.232131 2918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:13.232523 2918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:13.233705 2918 hybrid_clock.cc:648] HybridClock initialized: now 1786149793233678 us; error 56 us; skew 500 ppm
I20260808 00:43:13.235494 2918 webserver.cc:460] Webserver started at http://127.20.105.126:41643/ using document root <none> and password file <none>
I20260808 00:43:13.235836 2918 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:13.235898 2918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:13.236127 2918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:13.237597 2918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "cfa6a2bcac194bc7987e182cc7c2853c"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "40cd1722739c1e8a66d9f796f761174b"
server_key_iv: "2ae3759f1072a26fd8f3f8304706f720"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.238058 2918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "cfa6a2bcac194bc7987e182cc7c2853c"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "40cd1722739c1e8a66d9f796f761174b"
server_key_iv: "2ae3759f1072a26fd8f3f8304706f720"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.239787 2918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:43:13.240886 2938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.241222 2918 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:13.241344 2918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "cfa6a2bcac194bc7987e182cc7c2853c"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "40cd1722739c1e8a66d9f796f761174b"
server_key_iv: "2ae3759f1072a26fd8f3f8304706f720"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.241443 2918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:13.248011 2918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:13.248498 2918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:13.248689 2918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:13.254218 2918 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:33063
I20260808 00:43:13.254261 3013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:33063 every 8 connection(s)
I20260808 00:43:13.254700 2918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:43:13.255620 3014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:43:13.259099 3014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c: Bootstrap starting.
I20260808 00:43:13.259833 3014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c: Neither blocks nor log segments found. Creating new log.
I20260808 00:43:13.260226 3014 log.cc:824] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:43:13.260294 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 2918
I20260808 00:43:13.260428 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:43:13.260665 20901 external_mini_cluster.cc:1428] Setting key 6ae73d0859b634a04cf3ddbcdd4b3d61
I20260808 00:43:13.261080 3014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c: No bootstrap required, opened a new log
I20260808 00:43:13.263622 3014 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa6a2bcac194bc7987e182cc7c2853c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33063 } }
I20260808 00:43:13.263826 3014 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:43:13.263875 3014 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa6a2bcac194bc7987e182cc7c2853c, State: Initialized, Role: FOLLOWER
I20260808 00:43:13.264083 3014 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa6a2bcac194bc7987e182cc7c2853c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33063 } }
I20260808 00:43:13.264192 3014 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:43:13.264226 3014 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:43:13.264243 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.261420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33394 (local address 127.20.105.126:33063)
0808 00:43:13.261713 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:13.261724 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:13.261762 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:43:13.262584 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:13.262596 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:13.262656 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:13.262840 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:13.263626 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:13.263630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:13.263630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:43:13.263636 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:13.263692 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:43:13.263708 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:13.263783 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:13.263903 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 00:43:13.263973 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":125,"threads_started":1}
I20260808 00:43:13.264284 3014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:43:13.265414 3014 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa6a2bcac194bc7987e182cc7c2853c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33063 } }
I20260808 00:43:13.265512 3014 leader_election.cc:302] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfa6a2bcac194bc7987e182cc7c2853c; no voters:
I20260808 00:43:13.265645 3014 leader_election.cc:288] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:43:13.265734 3019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:43:13.265936 3019 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [term 1 LEADER]: Becoming Leader. State: Replica: cfa6a2bcac194bc7987e182cc7c2853c, State: Running, Role: LEADER
I20260808 00:43:13.266006 3014 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:43:13.266042 3019 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa6a2bcac194bc7987e182cc7c2853c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33063 } }
I20260808 00:43:13.266552 3022 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfa6a2bcac194bc7987e182cc7c2853c. Latest consensus state: current_term: 1 leader_uuid: "cfa6a2bcac194bc7987e182cc7c2853c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa6a2bcac194bc7987e182cc7c2853c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33063 } } }
I20260808 00:43:13.266666 3022 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [sys.catalog]: This master's current role is: LEADER
I20260808 00:43:13.266690 3020 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfa6a2bcac194bc7987e182cc7c2853c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa6a2bcac194bc7987e182cc7c2853c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33063 } } }
I20260808 00:43:13.266784 3020 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c [sys.catalog]: This master's current role is: LEADER
I20260808 00:43:13.266912 3025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:43:13.267508 3025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:43:13.269328 3025 catalog_manager.cc:1347] Generated new cluster ID: b1af868d6d6b4cd08dedd352d25891e5
I20260808 00:43:13.269402 3025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:43:13.275362 3025 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:43:13.275586 3025 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:43:13.282389 3025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfa6a2bcac194bc7987e182cc7c2853c: Generated new TSK 0
I20260808 00:43:13.282662 3025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:43:13.286031 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33063
--builtin_ntp_servers=127.20.105.84:40179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:43:13.399477 3050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:13.399780 3050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:13.399821 3050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:13.401145 3050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:13.401221 3050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:13.401268 3050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:43:13.402702 3050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:13.403026 3050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:13.403395 3050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:13.410395 3057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:13.410439 3059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:13.410449 3062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:13.410815 3050 server_base.cc:1034] running on GCE node
I20260808 00:43:13.411149 3050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:13.411478 3050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:13.412681 3050 hybrid_clock.cc:648] HybridClock initialized: now 1786149793412654 us; error 46 us; skew 500 ppm
I20260808 00:43:13.414412 3050 webserver.cc:460] Webserver started at http://127.20.105.65:46697/ using document root <none> and password file <none>
I20260808 00:43:13.414754 3050 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:13.414808 3050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:13.414995 3050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:13.416404 3050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "f73d9fab15db46449670292d84b26ddf"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "95d906797f138c5bf6499509168daed7"
server_key_iv: "d21eaccfa386e68c968fabaac052bc44"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.416834 3050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "f73d9fab15db46449670292d84b26ddf"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "95d906797f138c5bf6499509168daed7"
server_key_iv: "d21eaccfa386e68c968fabaac052bc44"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.418401 3050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:43:13.419309 3068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.419548 3050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:13.419664 3050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "f73d9fab15db46449670292d84b26ddf"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "95d906797f138c5bf6499509168daed7"
server_key_iv: "d21eaccfa386e68c968fabaac052bc44"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.419766 3050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:13.423576 3050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:13.423941 3050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:13.424149 3050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:13.424417 3050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:13.424815 3050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:13.424875 3050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.424917 3050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:13.424952 3050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.432264 3050 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:42731
I20260808 00:43:13.432756 3050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:43:13.432933 3231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:42731 every 8 connection(s)
I20260808 00:43:13.435205 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 3050
I20260808 00:43:13.435350 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:43:13.435572 20901 external_mini_cluster.cc:1428] Setting key bff32c535539a671dc63bf233ca784fd
I20260808 00:43:13.436914 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.433762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:54131 (local address 127.20.105.126:33063)
0808 00:43:13.433884 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:13.433886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:13.434163 (+ 277us) server_negotiation.cc:407] Connection header received
0808 00:43:13.434814 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:13.434815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:13.434824 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:13.434896 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:13.436574 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:13.436576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:13.436576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:43:13.436582 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:13.436605 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:43:13.436610 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:13.436683 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:13.436766 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 00:43:13.436785 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 00:43:13.436986 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33063
--builtin_ntp_servers=127.20.105.84:40179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 00:43:13.437077 3234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.433725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33063 (local address 127.20.105.65:54131)
0808 00:43:13.434003 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:13.434051 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:13.434682 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:13.435010 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:13.435014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:13.435025 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:13.436347 (+ 1322us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:43:13.436348 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:13.436386 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:43:13.436448 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:13.436683 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:13.436687 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:13.436690 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:43:13.436754 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 00:43:13.436822 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":111,"threads_started":1}
I20260808 00:43:13.437806 3232 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33063
I20260808 00:43:13.437948 3232 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:13.438144 3232 heartbeater.cc:511] Master 127.20.105.126:33063 requested a full tablet report, sending...
I20260808 00:43:13.438822 2963 ts_manager.cc:194] Registered new tserver with Master: f73d9fab15db46449670292d84b26ddf (127.20.105.65:42731)
W20260808 00:43:13.539232 3235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:13.539567 3235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:13.539609 3235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:13.541072 3235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:13.541162 3235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:13.541213 3235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:43:13.542832 3235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3235
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:13.543203 3235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:13.543613 3235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:13.550060 3246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:13.550169 3243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:13.550243 3235 server_base.cc:1034] running on GCE node
W20260808 00:43:13.550076 3242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:13.550665 3235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:13.551023 3235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:13.552212 3235 hybrid_clock.cc:648] HybridClock initialized: now 1786149793552198 us; error 60 us; skew 500 ppm
I20260808 00:43:13.553800 3235 webserver.cc:460] Webserver started at http://127.20.105.66:39321/ using document root <none> and password file <none>
I20260808 00:43:13.554127 3235 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:13.554207 3235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:13.554373 3235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:13.555771 3235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "d75b4d715aa54fcdbb2443c37634cfb3"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "8c8fb9fb667b0b3534d225048d8f3b19"
server_key_iv: "10e6c2cafe842247d5307035c90183a2"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.556304 3235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "d75b4d715aa54fcdbb2443c37634cfb3"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "8c8fb9fb667b0b3534d225048d8f3b19"
server_key_iv: "10e6c2cafe842247d5307035c90183a2"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.557915 3235 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:43:13.558830 3258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.559074 3235 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:43:13.559197 3235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "d75b4d715aa54fcdbb2443c37634cfb3"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "8c8fb9fb667b0b3534d225048d8f3b19"
server_key_iv: "10e6c2cafe842247d5307035c90183a2"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.559288 3235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:13.563395 3235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:13.563747 3235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:13.563910 3235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:13.564213 3235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:13.564611 3235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:13.564670 3235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.564713 3235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:13.564741 3235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.572137 3235 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:38545
I20260808 00:43:13.572182 3414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:38545 every 8 connection(s)
I20260808 00:43:13.572646 3235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:43:13.575619 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 3235
I20260808 00:43:13.575733 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:43:13.576076 20901 external_mini_cluster.cc:1428] Setting key a6a593d14c51211f1ef80f2ea7a51133
I20260808 00:43:13.576820 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.573703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:56547 (local address 127.20.105.126:33063)
0808 00:43:13.573817 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:13.573819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:13.574115 (+ 296us) server_negotiation.cc:407] Connection header received
0808 00:43:13.574694 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:13.574695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:13.574704 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:13.574781 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:13.576489 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:13.576492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:13.576492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:43:13.576499 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:13.576522 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:43:13.576528 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:13.576618 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:13.576724 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 00:43:13.576740 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 00:43:13.577162 3419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.573686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33063 (local address 127.20.105.66:56547)
0808 00:43:13.573939 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:13.573991 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:13.574594 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:13.574848 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:13.574852 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:13.574862 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:13.576238 (+ 1376us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:43:13.576239 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:13.576279 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:43:13.576338 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:13.576626 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:13.576634 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:13.576637 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:43:13.576729 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:43:13.576810 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":127,"threads_started":1}
I20260808 00:43:13.577505 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33063
--builtin_ntp_servers=127.20.105.84:40179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 00:43:13.577757 3415 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33063
I20260808 00:43:13.577893 3415 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:13.578130 3415 heartbeater.cc:511] Master 127.20.105.126:33063 requested a full tablet report, sending...
I20260808 00:43:13.578631 2963 ts_manager.cc:194] Registered new tserver with Master: d75b4d715aa54fcdbb2443c37634cfb3 (127.20.105.66:38545)
W20260808 00:43:13.678272 3420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:13.678544 3420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:13.678570 3420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:13.679885 3420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:13.679942 3420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:13.679973 3420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:43:13.681535 3420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33063
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:13.681895 3420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:13.682312 3420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:13.688560 3437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:13.688592 3431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:13.688676 3420 server_base.cc:1034] running on GCE node
W20260808 00:43:13.688905 3433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:13.689216 3420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:13.689590 3420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:13.690784 3420 hybrid_clock.cc:648] HybridClock initialized: now 1786149793690755 us; error 54 us; skew 500 ppm
I20260808 00:43:13.692536 3420 webserver.cc:460] Webserver started at http://127.20.105.67:35981/ using document root <none> and password file <none>
I20260808 00:43:13.692852 3420 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:13.692937 3420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:13.693105 3420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:13.694402 3420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "1dc934a5df08478d91c7414b9ef4143d"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "c864d8320dcf48ee16dfb3c652b73096"
server_key_iv: "24877c02ffd07fdf3e96fb8b86c8023f"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.694833 3420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "1dc934a5df08478d91c7414b9ef4143d"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "c864d8320dcf48ee16dfb3c652b73096"
server_key_iv: "24877c02ffd07fdf3e96fb8b86c8023f"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.696465 3420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 00:43:13.697366 3447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.697620 3420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:13.697747 3420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "1dc934a5df08478d91c7414b9ef4143d"
format_stamp: "Formatted at 2026-08-08 00:43:13 on dist-test-slave-kvfs"
server_key: "c864d8320dcf48ee16dfb3c652b73096"
server_key_iv: "24877c02ffd07fdf3e96fb8b86c8023f"
server_key_version: "encryptionkey@0"
I20260808 00:43:13.697831 3420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:13.701853 3420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:13.702172 3420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:13.702322 3420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:13.702549 3420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:13.702899 3420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:13.702955 3420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.702997 3420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:13.703039 3420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:13.710278 3420 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:39639
I20260808 00:43:13.710335 3607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:39639 every 8 connection(s)
I20260808 00:43:13.710748 3420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:43:13.714434 3018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.711667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:34683 (local address 127.20.105.126:33063)
0808 00:43:13.711754 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:13.711757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:13.712111 (+ 354us) server_negotiation.cc:407] Connection header received
0808 00:43:13.712749 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:13.712751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:13.712759 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:13.712819 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:13.714197 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:13.714199 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:13.714199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:43:13.714204 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:13.714222 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:43:13.714228 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:13.714278 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:13.714344 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 00:43:13.714359 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:43:13.714598 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:13.711646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33063 (local address 127.20.105.67:34683)
0808 00:43:13.711903 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:13.711947 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:13.712614 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:13.712885 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:13.712889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:13.712900 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:13.714030 (+ 1130us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:43:13.714031 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:13.714066 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:43:13.714101 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:13.714277 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:13.714279 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:13.714283 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:43:13.714340 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:43:13.714398 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":122,"threads_started":1}
I20260808 00:43:13.715164 3608 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33063
I20260808 00:43:13.715278 3608 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:13.715456 3608 heartbeater.cc:511] Master 127.20.105.126:33063 requested a full tablet report, sending...
I20260808 00:43:13.715866 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 3420
I20260808 00:43:13.715905 2963 ts_manager.cc:194] Registered new tserver with Master: 1dc934a5df08478d91c7414b9ef4143d (127.20.105.67:39639)
I20260808 00:43:13.715999 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:43:13.716316 20901 external_mini_cluster.cc:1428] Setting key e24ef21827e562c43cf599ec789d1abc
I20260808 00:43:13.717332 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:43:13.719581 3018 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:43:13.719006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.20.105.126:33063)
0808 00:43:13.719052 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:13.719054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:13.719188 (+ 134us) server_negotiation.cc:407] Connection header received
0808 00:43:13.719270 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:13.719271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:13.719277 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:13.719324 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:13.719496 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33396: BlockingRecv error: recv got EOF from 127.0.0.1:33396 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:43:13.719667 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 3050
I20260808 00:43:13.734241 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 3235
I20260808 00:43:13.748756 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 3420
I20260808 00:43:13.771440 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 2918
2026-08-08T00:43:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (690 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:43:13 dist-test-slave-kvfs krb5kdc[3619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:43:13 dist-test-slave-kvfs krb5kdc[3619](info): set up 2 sockets
Aug 08 00:43:13 dist-test-slave-kvfs krb5kdc[3619](info): commencing operation
krb5kdc: starting...
W20260808 00:43:15.071887 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:43:15 dist-test-slave-kvfs krb5kdc[3619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:43:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:43:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:43:15.221906 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:43233
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:43233
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:43:15.324626 3641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:15.324991 3641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:15.325034 3641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:15.326871 3641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:43:15.327044 3641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:15.327113 3641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:15.327140 3641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:43:15.327169 3641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:43:15.329414 3641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:43233
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:43233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:15.329826 3641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:15.330158 3641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:15.336417 3652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:15.336455 3648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:15.336436 3649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:15.336596 3641 server_base.cc:1034] running on GCE node
I20260808 00:43:15.337064 3641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:15.337425 3641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:15.338611 3641 hybrid_clock.cc:648] HybridClock initialized: now 1786149795338585 us; error 44 us; skew 500 ppm
Aug 08 00:43:15 dist-test-slave-kvfs krb5kdc[3619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:43:15.347415 3641 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:43:15.348219 3641 webserver.cc:460] Webserver started at http://127.20.105.126:38233/ using document root <none> and password file <none>
I20260808 00:43:15.348555 3641 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:15.348637 3641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:15.348811 3641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:15.350271 3641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "77a51329c1e54982a2cf7dfa7de314cc"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "ff554d8186304f8c0167ccebaf684740"
server_key_iv: "a0a7197453223ec5e4ba38239a8d7205"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.350714 3641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "77a51329c1e54982a2cf7dfa7de314cc"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "ff554d8186304f8c0167ccebaf684740"
server_key_iv: "a0a7197453223ec5e4ba38239a8d7205"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.352485 3641 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:43:15.353502 3661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:15.353765 3641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:15.353892 3641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "77a51329c1e54982a2cf7dfa7de314cc"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "ff554d8186304f8c0167ccebaf684740"
server_key_iv: "a0a7197453223ec5e4ba38239a8d7205"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.353984 3641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:15.372707 3641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:15.375633 3641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:15.375844 3641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:15.381398 3641 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:43233
I20260808 00:43:15.381446 3740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:43233 every 8 connection(s)
I20260808 00:43:15.381920 3641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:43:15.383206 3741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:43:15.386209 3741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc: Bootstrap starting.
I20260808 00:43:15.386936 3741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc: Neither blocks nor log segments found. Creating new log.
I20260808 00:43:15.387224 3741 log.cc:824] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc: Log is configured to *not* fsync() on all Append() calls
I20260808 00:43:15.388293 3741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc: No bootstrap required, opened a new log
I20260808 00:43:15.390390 3741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a51329c1e54982a2cf7dfa7de314cc" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43233 } }
I20260808 00:43:15.390602 3741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:43:15.390633 3741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a51329c1e54982a2cf7dfa7de314cc, State: Initialized, Role: FOLLOWER
I20260808 00:43:15.390574 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 3641
I20260808 00:43:15.390733 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:43:15.390851 3741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a51329c1e54982a2cf7dfa7de314cc" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43233 } }
I20260808 00:43:15.390959 3741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:43:15.391022 3741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:43:15.391038 20901 external_mini_cluster.cc:1428] Setting key d57f67abac1a65a62b4de6c185426d6a
I20260808 00:43:15.391088 3741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:43:15.392712 3741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a51329c1e54982a2cf7dfa7de314cc" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43233 } }
I20260808 00:43:15.392863 3741 leader_election.cc:302] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77a51329c1e54982a2cf7dfa7de314cc; no voters:
I20260808 00:43:15.393132 3741 leader_election.cc:288] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:43:15.393160 3747 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:43:15.393304 3747 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [term 1 LEADER]: Becoming Leader. State: Replica: 77a51329c1e54982a2cf7dfa7de314cc, State: Running, Role: LEADER
I20260808 00:43:15.393412 3747 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a51329c1e54982a2cf7dfa7de314cc" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43233 } }
I20260808 00:43:15.393497 3741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:43:15.393846 3747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77a51329c1e54982a2cf7dfa7de314cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a51329c1e54982a2cf7dfa7de314cc" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43233 } } }
I20260808 00:43:15.393877 3748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77a51329c1e54982a2cf7dfa7de314cc. Latest consensus state: current_term: 1 leader_uuid: "77a51329c1e54982a2cf7dfa7de314cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a51329c1e54982a2cf7dfa7de314cc" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43233 } } }
I20260808 00:43:15.394057 3747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [sys.catalog]: This master's current role is: LEADER
I20260808 00:43:15.394065 3748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc [sys.catalog]: This master's current role is: LEADER
I20260808 00:43:15.394377 3757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:43:15.395197 3757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:43:15.397653 3757 catalog_manager.cc:1347] Generated new cluster ID: 71cd811f88fe4e7c89fdf4480748d723
I20260808 00:43:15.397723 3757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:43:15 dist-test-slave-kvfs krb5kdc[3619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:43:15.405288 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:15.391821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35904 (local address 127.20.105.126:43233)
0808 00:43:15.392172 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:15.392187 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:15.392243 (+ 56us) server_negotiation.cc:407] Connection header received
0808 00:43:15.392417 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:15.392426 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:15.392511 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:15.392711 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:15.393796 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.395621 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.396839 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.397518 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.397666 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:15.400967 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:15.400972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:15.400972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:43:15.400988 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:15.403194 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:15.403689 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:15.403692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:15.403693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:15.403828 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:15.404158 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:15.404159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:15.404160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:15.404496 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:15.404733 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:15.404832 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 00:43:15.405043 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":149,"threads_started":1}
I20260808 00:43:15.409081 3757 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:43:15.409693 3757 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:43:15.420742 3757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc: Generated new TSK 0
I20260808 00:43:15.421051 3757 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:43:15.465382 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43233
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:43:15.570004 3775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:15.570312 3775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:15.570341 3775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:15.571662 3775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:15.571708 3775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:15.571738 3775 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:43:15.573601 3775 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:15.573863 3775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:15.574111 3775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:15.580322 3785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:15.580255 3780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:15.580236 3781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:15.580559 3775 server_base.cc:1034] running on GCE node
I20260808 00:43:15.580848 3775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:15.581205 3775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:15.582401 3775 hybrid_clock.cc:648] HybridClock initialized: now 1786149795582384 us; error 46 us; skew 500 ppm
Aug 08 00:43:15 dist-test-slave-kvfs krb5kdc[3619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:43:15.585978 3775 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:43:15.586613 3775 webserver.cc:460] Webserver started at http://127.20.105.65:35121/ using document root <none> and password file <none>
I20260808 00:43:15.586890 3775 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:15.586967 3775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:15.587139 3775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:15.588604 3775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "a9f46263dbc84d24a062b0e796a82ee8"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "8b2dedfb8b90d583da16d472ca2b82a1"
server_key_iv: "8b1f4f746729cafe5d772a9af73bfa1a"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.589079 3775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "a9f46263dbc84d24a062b0e796a82ee8"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "8b2dedfb8b90d583da16d472ca2b82a1"
server_key_iv: "8b1f4f746729cafe5d772a9af73bfa1a"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.590704 3775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:43:15.591624 3799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:15.591853 3775 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:15.591977 3775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "a9f46263dbc84d24a062b0e796a82ee8"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "8b2dedfb8b90d583da16d472ca2b82a1"
server_key_iv: "8b1f4f746729cafe5d772a9af73bfa1a"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.592134 3775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:15.611901 3775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:15.614967 3775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:15.615180 3775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:15.615527 3775 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:15.615942 3775 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:15.616003 3775 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:15.616065 3775 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:15.616118 3775 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:15.623567 3775 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:46249
I20260808 00:43:15.623618 3965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:46249 every 8 connection(s)
I20260808 00:43:15.624166 3775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:43:15 dist-test-slave-kvfs krb5kdc[3619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:43:15.634342 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 3775
I20260808 00:43:15.634490 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:43:15.634779 20901 external_mini_cluster.cc:1428] Setting key a107c7d1a1baffa9f03cfe58e001a88b
I20260808 00:43:15.635915 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:15.625168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:53965 (local address 127.20.105.126:43233)
0808 00:43:15.625251 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:15.625253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:15.625467 (+ 214us) server_negotiation.cc:407] Connection header received
0808 00:43:15.626139 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:15.626141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:15.626154 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:15.626237 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:15.627468 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.628154 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.629302 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.629545 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.629615 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:15.632488 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:15.632489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:15.632489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:43:15.632497 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:15.634031 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:15.634658 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:15.634659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:15.634660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:15.634722 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:15.635047 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:15.635048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:15.635049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:15.635217 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:15.635311 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:15.635698 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 00:43:15.635801 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:43:15.636220 3969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:15.625134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43233 (local address 127.20.105.65:53965)
0808 00:43:15.625336 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:15.625382 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:15.626014 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:15.626329 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:15.626333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:15.627073 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:15.627322 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:15.627327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.628250 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.628251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:15.629187 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:15.629190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.629657 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.629658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:15.629801 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:15.630548 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:15.630560 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:15.632350 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:15.634143 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:43:15.634154 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:43:15.634157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:43:15.634556 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:43:15.634804 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:43:15.634805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:43:15.634805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:43:15.634977 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:43:15.635317 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:15.635326 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:15.635545 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 00:43:15.635676 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:43:15.635882 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
I20260808 00:43:15.636848 3967 heartbeater.cc:347] Connected to a master server at 127.20.105.126:43233
I20260808 00:43:15.637012 3967 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:15.637271 3967 heartbeater.cc:511] Master 127.20.105.126:43233 requested a full tablet report, sending...
I20260808 00:43:15.638069 3688 ts_manager.cc:194] Registered new tserver with Master: a9f46263dbc84d24a062b0e796a82ee8 (127.20.105.65:46249)
I20260808 00:43:15.638957 3688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:53965
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:43:15.671263 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43233
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:43:15.771106 3972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:15.771386 3972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:15.771416 3972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:15.772833 3972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:15.772890 3972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:15.772919 3972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:43:15.774519 3972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:15.774829 3972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:15.775128 3972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:15.781670 3979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:15.781670 3982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:15.781728 3972 server_base.cc:1034] running on GCE node
W20260808 00:43:15.781683 3980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:15.782189 3972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:15.782447 3972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:15.783600 3972 hybrid_clock.cc:648] HybridClock initialized: now 1786149795783577 us; error 42 us; skew 500 ppm
Aug 08 00:43:15 dist-test-slave-kvfs krb5kdc[3619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:43:15.786794 3972 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:43:15.787344 3972 webserver.cc:460] Webserver started at http://127.20.105.66:44529/ using document root <none> and password file <none>
I20260808 00:43:15.787658 3972 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:15.787734 3972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:15.787910 3972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:15.789304 3972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "c61b51a9cbcd4f6c9b06be183aabdb93"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "5ed64644fe7c7f769a7f3af391877499"
server_key_iv: "9d232072311bd2c71f8f87e52698e46a"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.789747 3972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "c61b51a9cbcd4f6c9b06be183aabdb93"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "5ed64644fe7c7f769a7f3af391877499"
server_key_iv: "9d232072311bd2c71f8f87e52698e46a"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.791256 3972 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:43:15.792265 3993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:15.792506 3972 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:15.792630 3972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "c61b51a9cbcd4f6c9b06be183aabdb93"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "5ed64644fe7c7f769a7f3af391877499"
server_key_iv: "9d232072311bd2c71f8f87e52698e46a"
server_key_version: "encryptionkey@0"
I20260808 00:43:15.792716 3972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:15.825706 3972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:15.828759 3972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:15.828965 3972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:15.829296 3972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:15.829754 3972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:15.829814 3972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:15.829864 3972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:15.829892 3972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:15.837087 3972 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:44797
I20260808 00:43:15.837137 4153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:44797 every 8 connection(s)
I20260808 00:43:15.837604 3972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:43:15.839943 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 3972
I20260808 00:43:15.840121 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:43:15.840392 20901 external_mini_cluster.cc:1428] Setting key 74fc6c6ed456555cb05510d9bbad5eb3
Aug 08 00:43:15 dist-test-slave-kvfs krb5kdc[3619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:43:15.851265 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:15.838660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:40691 (local address 127.20.105.126:43233)
0808 00:43:15.838781 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:15.838783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:15.839003 (+ 220us) server_negotiation.cc:407] Connection header received
0808 00:43:15.839721 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:15.839723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:15.839740 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:15.839827 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:15.841386 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.842147 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.843358 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.843690 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.843779 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:15.847057 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:15.847059 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:15.847059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:43:15.847071 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:15.849207 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:15.849749 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:15.849751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:15.849752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:15.849847 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:15.850216 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:15.850218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:15.850218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:15.850443 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:15.850551 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:15.850968 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 00:43:15.851097 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 00:43:15.851328 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:15.838623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43233 (local address 127.20.105.66:40691)
0808 00:43:15.838814 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:15.838866 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:15.839548 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:15.839953 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:15.839960 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:15.840942 (+ 982us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:15.841275 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:15.841282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.842272 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.842274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:15.843257 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:15.843261 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:15.843817 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:15.843819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:15.844063 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:15.844747 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:15.844761 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:15.846913 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:15.849331 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:43:15.849341 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:43:15.849344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:43:15.849651 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:43:15.849955 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:43:15.849957 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:43:15.849957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:43:15.850116 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:43:15.850586 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:15.850594 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:15.850776 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 00:43:15.850909 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 00:43:15.851100 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
I20260808 00:43:15.852417 4154 heartbeater.cc:347] Connected to a master server at 127.20.105.126:43233
I20260808 00:43:15.852583 4154 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:15.852878 4154 heartbeater.cc:511] Master 127.20.105.126:43233 requested a full tablet report, sending...
I20260808 00:43:15.853544 3688 ts_manager.cc:194] Registered new tserver with Master: c61b51a9cbcd4f6c9b06be183aabdb93 (127.20.105.66:44797)
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
I20260808 00:43:15.854090 3688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:40691
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:43:15.880435 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43233
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:43:15.981468 4161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:43:15.981763 4161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:43:15.981796 4161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:43:15.983220 4161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:43:15.983287 4161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:43:15.983322 4161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:43:15.985193 4161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:43233
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:43:15.985543 4161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:43:15.985873 4161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:43:15.992331 4172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:15.992331 4169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:43:15.992332 4168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:43:15.992687 4161 server_base.cc:1034] running on GCE node
I20260808 00:43:15.992990 4161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:43:15.993301 4161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:43:15.994501 4161 hybrid_clock.cc:648] HybridClock initialized: now 1786149795994476 us; error 43 us; skew 500 ppm
Aug 08 00:43:15 dist-test-slave-kvfs krb5kdc[3619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:43:15.998195 4161 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:43:15.998737 4161 webserver.cc:460] Webserver started at http://127.20.105.67:36519/ using document root <none> and password file <none>
I20260808 00:43:15.999070 4161 fs_manager.cc:359] Metadata directory not provided
I20260808 00:43:15.999125 4161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:43:15.999287 4161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:43:16.000761 4161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "e0aa07d6d59f45d18bdd49d30c6f5942"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "49a0dc0f76b87f029fed2d25b61891be"
server_key_iv: "b6180c4a576e49f85819b582b2e4d598"
server_key_version: "encryptionkey@0"
I20260808 00:43:16.001243 4161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "e0aa07d6d59f45d18bdd49d30c6f5942"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "49a0dc0f76b87f029fed2d25b61891be"
server_key_iv: "b6180c4a576e49f85819b582b2e4d598"
server_key_version: "encryptionkey@0"
I20260808 00:43:16.003088 4161 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 00:43:16.004150 4184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:16.004361 4161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:43:16.004472 4161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "e0aa07d6d59f45d18bdd49d30c6f5942"
format_stamp: "Formatted at 2026-08-08 00:43:15 on dist-test-slave-kvfs"
server_key: "49a0dc0f76b87f029fed2d25b61891be"
server_key_iv: "b6180c4a576e49f85819b582b2e4d598"
server_key_version: "encryptionkey@0"
I20260808 00:43:16.004573 4161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:43:16.012732 4161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:43:16.015285 4161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:43:16.015461 4161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:43:16.015760 4161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:43:16.016232 4161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:43:16.016273 4161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:16.016301 4161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:43:16.016314 4161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:43:16.023589 4161 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:44745
I20260808 00:43:16.023631 4345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:44745 every 8 connection(s)
I20260808 00:43:16.024135 4161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:43:16.028662 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 4161
I20260808 00:43:16.028793 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:43:16.029008 20901 external_mini_cluster.cc:1428] Setting key 638af6255c925528b5c7070f9c32bb94
Aug 08 00:43:16 dist-test-slave-kvfs krb5kdc[3619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149795, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:43:16.035879 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.025270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:42681 (local address 127.20.105.126:43233)
0808 00:43:16.025377 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:16.025380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:16.025555 (+ 175us) server_negotiation.cc:407] Connection header received
0808 00:43:16.026227 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:16.026229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:16.026246 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:16.026319 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:16.027529 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.028171 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.029785 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.030042 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.030129 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.032782 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:16.032785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:16.032786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:43:16.032794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:16.034199 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:16.034681 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:16.034682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:16.034683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:16.034754 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:16.035070 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:16.035071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:16.035071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:16.035217 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:16.035304 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:16.035625 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 00:43:16.035737 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 00:43:16.036140 4349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.025144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43233 (local address 127.20.105.67:42681)
0808 00:43:16.025351 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:16.025416 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:16.026070 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:16.026422 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:16.026427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:16.027146 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:43:16.027404 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.027409 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.028283 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.028284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.029642 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.029646 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.030139 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.030141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.030310 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.030882 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:43:16.030892 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:43:16.032666 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:43:16.034320 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:43:16.034328 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:43:16.034330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:43:16.034592 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:43:16.034828 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:43:16.034829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:43:16.034830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:43:16.034992 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:43:16.035291 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:43:16.035299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:43:16.035473 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 00:43:16.035599 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 00:43:16.035797 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260808 00:43:16.036803 4346 heartbeater.cc:347] Connected to a master server at 127.20.105.126:43233
I20260808 00:43:16.036940 4346 heartbeater.cc:464] Registering TS with master...
I20260808 00:43:16.037211 4346 heartbeater.cc:511] Master 127.20.105.126:43233 requested a full tablet report, sending...
I20260808 00:43:16.037812 3688 ts_manager.cc:194] Registered new tserver with Master: e0aa07d6d59f45d18bdd49d30c6f5942 (127.20.105.67:44745)
I20260808 00:43:16.038268 3688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:42681
I20260808 00:43:16.041177 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:43:16.050495 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.042846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35912 (local address 127.20.105.126:43233)
0808 00:43:16.043037 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:16.043039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:16.043144 (+ 105us) server_negotiation.cc:407] Connection header received
0808 00:43:16.043270 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:16.043271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:16.043284 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:16.043381 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:43:16.044095 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.044634 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.045472 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.045683 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.045780 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.047307 (+ 1527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:43:16.047309 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:43:16.047309 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:43:16.047318 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:43:16.048914 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:16.049453 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:16.049455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:16.049455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:16.049534 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:43:16.049773 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:43:16.049775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:43:16.049775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:43:16.049984 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:43:16.050130 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:16.050257 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 00:43:16.050360 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260808 00:43:16.052687 3688 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35912:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:43:16.054785 3688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:43:16.062922 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.057776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.20.105.67:44745)
0808 00:43:16.058251 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:16.058256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:16.058268 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:43:16.058317 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:16.058320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:16.058337 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:16.058454 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:43:16.058727 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.060004 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.061544 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.062399 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.062509 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.062582 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:16.062751 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 00:43:16.062807 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":106,"threads_started":1}
I20260808 00:43:16.062899 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.057676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44745 (local address 127.0.0.1:35898)
0808 00:43:16.057931 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:16.057986 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:16.058072 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:16.058457 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:16.058460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:16.058462 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:43:16.058646 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.058649 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.060147 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.060148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.061446 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.061449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.062513 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.062514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.062617 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.062635 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:43:16.062716 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 00:43:16.062785 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":109,"threads_started":1}
I20260808 00:43:16.063971 4365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.057680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46249 (local address 127.0.0.1:51156)
0808 00:43:16.058154 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:16.058167 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:16.058236 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:16.058733 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:16.058734 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:16.058736 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:43:16.058790 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.058792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.061570 (+ 2778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.061571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.062667 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.062670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.063694 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.063695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.063763 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.063769 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:43:16.063829 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:43:16.063878 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
I20260808 00:43:16.063989 4369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.057850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.20.105.65:46249)
0808 00:43:16.058128 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:16.058134 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:16.058253 (+ 119us) server_negotiation.cc:407] Connection header received
0808 00:43:16.058334 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:16.058336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:16.058355 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:16.058554 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:43:16.058877 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.060013 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.062788 (+ 2775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.063586 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.063690 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.063760 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:16.063847 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 00:43:16.063896 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":167,"threads_started":1}
I20260808 00:43:16.064301 4370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.058007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:44797 (local address 127.0.0.1:49238)
0808 00:43:16.058658 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:16.058668 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:16.058726 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:16.060485 (+ 1759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:16.060487 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:16.060491 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:43:16.060554 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.060557 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.061612 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.061613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.062377 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.062380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.064042 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.064044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.064110 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.064115 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:43:16.064174 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:43:16.064224 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":117,"threads_started":1}
I20260808 00:43:16.064316 4249 tablet_service.cc:1467] Processing CreateTablet for tablet c983de09133c4019bb85950c75f8144c (DEFAULT_TABLE table=test-table [id=e18f76e86be54e21b5259d326a57a941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:43:16.064658 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.058335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.20.105.66:44797)
0808 00:43:16.060236 (+ 1901us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:16.060241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:16.060253 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:43:16.060299 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:16.060301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:16.060319 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:16.060424 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:43:16.060645 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.061479 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.062489 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.063665 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.063735 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.063792 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:16.064495 (+ 703us) server_negotiation.cc:299] Negotiation successful
0808 00:43:16.064565 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1850,"thread_start_us":122,"threads_started":1}
I20260808 00:43:16.064867 4249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c983de09133c4019bb85950c75f8144c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:43:16.065124 3874 tablet_service.cc:1467] Processing CreateTablet for tablet c983de09133c4019bb85950c75f8144c (DEFAULT_TABLE table=test-table [id=e18f76e86be54e21b5259d326a57a941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:43:16.065353 4058 tablet_service.cc:1467] Processing CreateTablet for tablet c983de09133c4019bb85950c75f8144c (DEFAULT_TABLE table=test-table [id=e18f76e86be54e21b5259d326a57a941]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:43:16.065488 3874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c983de09133c4019bb85950c75f8144c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:43:16.065723 4058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c983de09133c4019bb85950c75f8144c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:43:16.068686 4373 tablet_bootstrap.cc:492] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8: Bootstrap starting.
I20260808 00:43:16.069455 4375 tablet_bootstrap.cc:492] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Bootstrap starting.
I20260808 00:43:16.069665 4373 tablet_bootstrap.cc:654] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8: Neither blocks nor log segments found. Creating new log.
I20260808 00:43:16.069792 4374 tablet_bootstrap.cc:492] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: Bootstrap starting.
I20260808 00:43:16.070310 4375 tablet_bootstrap.cc:654] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Neither blocks nor log segments found. Creating new log.
I20260808 00:43:16.070468 4374 tablet_bootstrap.cc:654] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: Neither blocks nor log segments found. Creating new log.
I20260808 00:43:16.070442 4373 log.cc:824] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8: Log is configured to *not* fsync() on all Append() calls
I20260808 00:43:16.071008 4375 log.cc:824] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Log is configured to *not* fsync() on all Append() calls
I20260808 00:43:16.071139 4374 log.cc:824] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: Log is configured to *not* fsync() on all Append() calls
I20260808 00:43:16.071542 4373 tablet_bootstrap.cc:492] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8: No bootstrap required, opened a new log
I20260808 00:43:16.071700 4373 ts_tablet_manager.cc:1397] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:43:16.071954 4374 tablet_bootstrap.cc:492] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: No bootstrap required, opened a new log
I20260808 00:43:16.072064 4374 ts_tablet_manager.cc:1397] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:43:16.073760 4373 raft_consensus.cc:348] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.073830 4374 raft_consensus.cc:348] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.073956 4374 raft_consensus.cc:374] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:43:16.073961 4373 raft_consensus.cc:374] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:43:16.073979 4374 raft_consensus.cc:729] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0aa07d6d59f45d18bdd49d30c6f5942, State: Initialized, Role: FOLLOWER
I20260808 00:43:16.074012 4373 raft_consensus.cc:729] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9f46263dbc84d24a062b0e796a82ee8, State: Initialized, Role: FOLLOWER
I20260808 00:43:16.074158 4373 consensus_queue.cc:260] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.074173 4374 consensus_queue.cc:260] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.074524 4374 ts_tablet_manager.cc:1428] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:43:16.074532 4373 ts_tablet_manager.cc:1428] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:43:16.074559 3967 heartbeater.cc:503] Master 127.20.105.126:43233 was elected leader, sending a full tablet report...
I20260808 00:43:16.074678 4346 heartbeater.cc:503] Master 127.20.105.126:43233 was elected leader, sending a full tablet report...
I20260808 00:43:16.075345 4375 tablet_bootstrap.cc:492] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: No bootstrap required, opened a new log
I20260808 00:43:16.075435 4375 ts_tablet_manager.cc:1397] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 00:43:16.077337 4375 raft_consensus.cc:348] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.077536 4375 raft_consensus.cc:374] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:43:16.077607 4375 raft_consensus.cc:729] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c61b51a9cbcd4f6c9b06be183aabdb93, State: Initialized, Role: FOLLOWER
I20260808 00:43:16.077755 4375 consensus_queue.cc:260] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.078068 4375 ts_tablet_manager.cc:1428] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:43:16.078081 4154 heartbeater.cc:503] Master 127.20.105.126:43233 was elected leader, sending a full tablet report...
W20260808 00:43:16.088279 4157 tablet.cc:2369] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:43:16.125252 3968 tablet.cc:2369] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:43:16.209123 4379 raft_consensus.cc:482] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:43:16.209331 4379 raft_consensus.cc:504] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.209869 4379 leader_election.cc:288] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0aa07d6d59f45d18bdd49d30c6f5942 (127.20.105.67:44745), c61b51a9cbcd4f6c9b06be183aabdb93 (127.20.105.66:44797)
I20260808 00:43:16.213729 4382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.210085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44745 (local address 127.20.105.65:55079)
0808 00:43:16.210249 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:16.210263 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:16.210383 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:16.210613 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:16.210614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:16.210616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:43:16.210728 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.210731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.211495 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.211496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.212640 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.212643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.213434 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.213435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.213513 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.213520 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:43:16.213578 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:43:16.213625 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260808 00:43:16.213716 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.210190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:55079 (local address 127.20.105.67:44745)
0808 00:43:16.210253 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:16.210255 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:16.210346 (+ 91us) server_negotiation.cc:407] Connection header received
0808 00:43:16.210467 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:16.210468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:16.210481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:16.210514 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:43:16.210796 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.211407 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.212757 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.213338 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.213428 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.213440 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:16.213588 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 00:43:16.213635 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:43:16.214530 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.210414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:50043 (local address 127.20.105.66:44797)
0808 00:43:16.210468 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:16.210470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:16.210610 (+ 140us) server_negotiation.cc:407] Connection header received
0808 00:43:16.210697 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:16.210698 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:16.210712 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:16.210766 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:43:16.211693 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.212322 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.213569 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.214159 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.214251 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.214267 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:16.214421 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 00:43:16.214465 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:43:16.214545 4383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.210326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:44797 (local address 127.20.105.65:50043)
0808 00:43:16.210529 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:16.210538 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:43:16.210625 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:43:16.211517 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:43:16.211518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:43:16.211520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:43:16.211598 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.211600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.212420 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.212421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.213379 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:43:16.213403 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.214254 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.214255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:43:16.214313 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.214317 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:43:16.214402 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 00:43:16.214453 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":61,"threads_started":1}
I20260808 00:43:16.214556 4279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c983de09133c4019bb85950c75f8144c" candidate_uuid: "a9f46263dbc84d24a062b0e796a82ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" is_pre_election: true
I20260808 00:43:16.214783 4279 raft_consensus.cc:2454] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9f46263dbc84d24a062b0e796a82ee8 in term 0.
I20260808 00:43:16.215099 3804 leader_election.cc:302] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9f46263dbc84d24a062b0e796a82ee8, e0aa07d6d59f45d18bdd49d30c6f5942; no voters:
I20260808 00:43:16.215061 4085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c983de09133c4019bb85950c75f8144c" candidate_uuid: "a9f46263dbc84d24a062b0e796a82ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" is_pre_election: true
I20260808 00:43:16.215246 4085 raft_consensus.cc:2454] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9f46263dbc84d24a062b0e796a82ee8 in term 0.
I20260808 00:43:16.215366 4379 raft_consensus.cc:2781] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:43:16.215461 4379 raft_consensus.cc:482] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:43:16.215487 4379 raft_consensus.cc:3037] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:43:16.216507 4379 raft_consensus.cc:504] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.216679 4379 leader_election.cc:288] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [CANDIDATE]: Term 1 election: Requested vote from peers e0aa07d6d59f45d18bdd49d30c6f5942 (127.20.105.67:44745), c61b51a9cbcd4f6c9b06be183aabdb93 (127.20.105.66:44797)
I20260808 00:43:16.216930 4085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c983de09133c4019bb85950c75f8144c" candidate_uuid: "a9f46263dbc84d24a062b0e796a82ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93"
I20260808 00:43:16.216905 4279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c983de09133c4019bb85950c75f8144c" candidate_uuid: "a9f46263dbc84d24a062b0e796a82ee8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942"
I20260808 00:43:16.217012 4085 raft_consensus.cc:3037] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:43:16.217043 4279 raft_consensus.cc:3037] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:43:16.217903 4085 raft_consensus.cc:2454] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9f46263dbc84d24a062b0e796a82ee8 in term 1.
I20260808 00:43:16.218026 4279 raft_consensus.cc:2454] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9f46263dbc84d24a062b0e796a82ee8 in term 1.
I20260808 00:43:16.218082 3804 leader_election.cc:302] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9f46263dbc84d24a062b0e796a82ee8, c61b51a9cbcd4f6c9b06be183aabdb93; no voters:
I20260808 00:43:16.218189 4379 raft_consensus.cc:2781] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:43:16.218410 4379 raft_consensus.cc:686] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 1 LEADER]: Becoming Leader. State: Replica: a9f46263dbc84d24a062b0e796a82ee8, State: Running, Role: LEADER
I20260808 00:43:16.218544 4379 consensus_queue.cc:237] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } }
I20260808 00:43:16.219741 3687 catalog_manager.cc:5495] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9f46263dbc84d24a062b0e796a82ee8 (127.20.105.65). New cstate: current_term: 1 leader_uuid: "a9f46263dbc84d24a062b0e796a82ee8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9f46263dbc84d24a062b0e796a82ee8" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 46249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 } health_report { overall_health: UNKNOWN } } }
I20260808 00:43:16.250487 4369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.247636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.20.105.65:46249)
0808 00:43:16.247716 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:43:16.247719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:43:16.247732 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:43:16.247810 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:43:16.247811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:43:16.247838 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:43:16.247923 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:43:16.248150 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.248692 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.249536 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:43:16.249765 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:43:16.249849 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:43:16.250061 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:43:16.250209 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:43:16.250323 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 00:43:16.250342 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 00:43:16.250370 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:43:16.253198 4085 raft_consensus.cc:1261] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 1 FOLLOWER]: Refusing update from remote peer a9f46263dbc84d24a062b0e796a82ee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:43:16.253199 4279 raft_consensus.cc:1261] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 1 FOLLOWER]: Refusing update from remote peer a9f46263dbc84d24a062b0e796a82ee8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:43:16.253723 4384 consensus_queue.cc:1035] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c61b51a9cbcd4f6c9b06be183aabdb93" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:43:16.253885 4379 consensus_queue.cc:1035] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0aa07d6d59f45d18bdd49d30c6f5942" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:43:16.259882 3687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:43:16.260076 3687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35912:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:43:16.261123 3687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77a51329c1e54982a2cf7dfa7de314cc: Sending DeleteTablet for 3 replicas of tablet c983de09133c4019bb85950c75f8144c
I20260808 00:43:16.261600 4058 tablet_service.cc:1514] Processing DeleteTablet for tablet c983de09133c4019bb85950c75f8144c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:43:16 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:49238
I20260808 00:43:16.261656 3874 tablet_service.cc:1514] Processing DeleteTablet for tablet c983de09133c4019bb85950c75f8144c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:43:16 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:51156
I20260808 00:43:16.261667 4249 tablet_service.cc:1514] Processing DeleteTablet for tablet c983de09133c4019bb85950c75f8144c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:43:16 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:35898
I20260808 00:43:16.261927 4408 tablet_replica.cc:330] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: stopping tablet replica
I20260808 00:43:16.262038 4409 tablet_replica.cc:330] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8: stopping tablet replica
I20260808 00:43:16.262106 4408 raft_consensus.cc:2229] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:43:16.262144 4411 tablet_replica.cc:330] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: stopping tablet replica
I20260808 00:43:16.262205 4409 raft_consensus.cc:2229] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:43:16.262272 4411 raft_consensus.cc:2229] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:43:16.262326 4408 raft_consensus.cc:2258] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:43:16.262374 4409 raft_consensus.cc:2258] T c983de09133c4019bb85950c75f8144c P a9f46263dbc84d24a062b0e796a82ee8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:43:16.262368 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 3775
I20260808 00:43:16.262423 4411 raft_consensus.cc:2258] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:43:16.266556 4408 ts_tablet_manager.cc:1905] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:43:16.269430 4408 ts_tablet_manager.cc:1918] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:43:16.269640 4408 log.cc:1196] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/c983de09133c4019bb85950c75f8144c
I20260808 00:43:16.270200 4408 ts_tablet_manager.cc:1939] T c983de09133c4019bb85950c75f8144c P c61b51a9cbcd4f6c9b06be183aabdb93: Deleting consensus metadata
I20260808 00:43:16.270925 4411 ts_tablet_manager.cc:1905] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:43:16.270978 3664 catalog_manager.cc:4841] TS c61b51a9cbcd4f6c9b06be183aabdb93 (127.20.105.66:44797): tablet c983de09133c4019bb85950c75f8144c (table test-table [id=e18f76e86be54e21b5259d326a57a941]) successfully deleted
I20260808 00:43:16.272662 4411 ts_tablet_manager.cc:1918] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:43:16.272771 4411 log.cc:1196] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/c983de09133c4019bb85950c75f8144c
I20260808 00:43:16.273092 4411 ts_tablet_manager.cc:1939] T c983de09133c4019bb85950c75f8144c P e0aa07d6d59f45d18bdd49d30c6f5942: Deleting consensus metadata
I20260808 00:43:16.273607 3664 catalog_manager.cc:4841] TS e0aa07d6d59f45d18bdd49d30c6f5942 (127.20.105.67:44745): tablet c983de09133c4019bb85950c75f8144c (table test-table [id=e18f76e86be54e21b5259d326a57a941]) successfully deleted
W20260808 00:43:16.288830 3663 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:46249 (error 108)
I20260808 00:43:16.289676 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 3972
I20260808 00:43:16.289716 4366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:43:16.289485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:46249 (local address 127.0.0.1:51178)
0808 00:43:16.289603 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 00:43:16.289667 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46249: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260808 00:43:16.305476 3663 catalog_manager.cc:4584] TS a9f46263dbc84d24a062b0e796a82ee8 (127.20.105.65:46249): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c983de09133c4019bb85950c75f8144c: Network error: Client connection negotiation failed: client connection to 127.20.105.65:46249: connect: Connection refused (error 111)
I20260808 00:43:16.312314 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 4161
I20260808 00:43:16.332859 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 3641
2026-08-08T00:43:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2570 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:43:16 dist-test-slave-kvfs krb5kdc[4416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:43:16 dist-test-slave-kvfs krb5kdc[4416](info): set up 2 sockets
Aug 08 00:43:16 dist-test-slave-kvfs krb5kdc[4416](info): commencing operation
krb5kdc: starting...
W20260808 00:43:17.617017 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:43:17 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:43:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:43:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/rangeradmin_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/rangeradmin_127.20.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/rangerlookup_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/rangerlookup_127.20.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/HTTP_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/HTTP_127.20.105.84@KRBTEST.COM.keytab.
I20260808 00:43:17.828692 20901 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzO4qSW/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/postgres -l logfile start
2026-08-08 00:43:20.778 UTC [4456] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 00:43:20.779 UTC [4456] LOG: listening on IPv4 address "127.20.105.84", port 41157
2026-08-08 00:43:20.782 UTC [4456] LOG: listening on Unix socket "/tmp/.s.PGSQL.41157"
2026-08-08 00:43:20.809 UTC [4460] LOG: database system was shut down at 2026-08-08 00:43:18 UTC
2026-08-08 00:43:20.815 UTC [4456] LOG: database system is ready to accept connections
I20260808 00:43:21.973167 20901 mini_postgres.cc:96] Postgres bound to 41157
2026-08-08 00:43:21.983 UTC [4469] FATAL: database "slave" does not exist
127.20.105.84:41157 - accepting connections
I20260808 00:43:21.984197 20901 mini_ranger.cc:161] Starting Ranger...
I20260808 00:43:22.003273 20901 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 00:43:22.621554 20901 mini_ranger.cc:87] Created ranger Postgres database
I20260808 00:43:22.621655 20901 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-admin
2026-08-08 00:43:22,968 [I] DB FLAVOR :POSTGRES
2026-08-08 00:43:22,969 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:43:22,969 [I] Checking connection..
2026-08-08 00:43:22,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 00:43:23,277 [I] Checking connection passed.
2026-08-08 00:43:23,277 [I] --------- Verifying version history table ---------
2026-08-08 00:43:23,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:43:23,580 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:43:23,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:43:23,900 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:43:23,901 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 00:43:23,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 00:43:24.201 UTC [4575] WARNING: there is no transaction in progress
2026-08-08 00:43:24,211 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 00:43:24,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:43:24,516 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 00:43:24,516 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:43:24,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 00:43:24,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 00:43:25,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-kvfs.c.gcp-upstream.internal','N') ;"
2026-08-08 00:43:25,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:43:25,718 [I] Table x_portal_user does not exist in database ranger
2026-08-08 00:43:25,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:43:26,024 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 00:43:26,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:43:26,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:43:26,612 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 00:43:26,613 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 00:43:27.813 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:27.834 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:27.856 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:27.877 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:27.899 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:27.921 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:27.988 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:27.997 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28.015 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28.028 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28.041 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28.058 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28.070 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28.079 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28.088 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28.096 UTC [4733] WARNING: there is no transaction in progress
2026-08-08 00:43:28,939 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 00:43:28,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 00:43:29,227 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 00:43:29,227 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:43:29,524 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 00:43:29,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:43:29,824 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 00:43:29,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:43:30,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:43:30,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-kvfs.c.gcp-upstream.internal';"
2026-08-08 00:43:30,731 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 00:43:30,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 00:43:31,033 [I] DB_PATCHES have already been applied
I20260808 00:43:31.040797 20901 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzO4qSW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 00:43:31.040891 20901 mini_ranger.cc:193] Using host: 127.20.105.84
I20260808 00:43:31.046197 20901 mini_ranger.cc:227] Ranger admin URL: http://127.20.105.84:58875
Aug 08, 2026 12:43:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:43:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:43:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 12:43:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 12:43:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 12:43:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 12:43:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 00:43:32.478587 20901 mini_ranger.cc:160] Time spent starting Ranger: real 10.494s user 0.003s sys 0.011s
Aug 08, 2026 12:43:56 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 12:43:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 12:43:56 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 12:43:56 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 12:43:57 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 00:43:58.006981 20901 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/rangerkms_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/rangerkms_127.20.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.105.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.105.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.105.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/HTTP_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.105.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/HTTP_127.20.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 00:43:58 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149838, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 00:43:58.117914 20901 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 00:43:58.134196 20901 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 00:43:59.170863 20901 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 00:43:59.170967 20901 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms
I20260808 00:43:59.170981 20901 mini_ranger_kms.cc:227] Using postgres at 127.20.105.84:41157
2026-08-08 00:43:59,696 [I] DB FLAVOR :POSTGRES
2026-08-08 00:43:59,696 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:43:59,696 [I] Checking connection
2026-08-08 00:43:59,696 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 00:44:00,044 [I] connection success
2026-08-08 00:44:00,044 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 00:44:00,044 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 00:44:00,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 00:44:00,357 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 00:44:00,358 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:44:00,358 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 00:44:00,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:41157/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 00:44:00,696 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 00:44:01.317736 20901 mini_ranger_kms.cc:310] Created kms service
I20260808 00:44:01.528481 20901 mini_ranger_kms.cc:326] Created kudu user
I20260808 00:44:01.582090 20901 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 00:44:01 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149838, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:44:02.038312 20901 mini_ranger_kms.cc:384] Added ranger policy
I20260808 00:44:02.038502 20901 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 00:44:02.038540 20901 mini_ranger_kms.cc:240] Using host: 127.20.105.84
I20260808 00:44:02.044523 20901 mini_ranger_kms.cc:276] Ranger KMS PID: 4980
I20260808 00:44:02.044699 20901 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.105.84:40753
Aug 08, 2026 12:44:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:44:03 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:44:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 12:44:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 12:44:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 12:44:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 00:44:04.846257 20901 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.728s user 0.000s sys 0.499s
I20260808 00:44:04.846413 20901 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 00:44:04.846518 20901 mini_ranger_kms.cc:401] 127.20.105.84:40753/kms/v1/keys
Aug 08 00:44:04 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.105.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 00:44:05 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.105.84@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
Aug 08 00:44:05 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149844, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.105.84@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
Aug 08 00:44:08 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:44:08.723227 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:32877
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:32877
--ranger_config_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:40753/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:44:08.846897 5059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:44:08.847321 5059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:44:08.847371 5059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:44:08.849076 5059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:44:08.849174 5059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:44:08.849200 5059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:44:08.849223 5059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:44:08.849254 5059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:44:08.851522 5059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.105.84:40753/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:32877
--ranger_config_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:32877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:44:08.852038 5059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:44:08.852393 5059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:44:08.858819 5070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:44:08.858771 5068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:44:08.858893 5067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:44:08.858915 5059 server_base.cc:1034] running on GCE node
I20260808 00:44:08.859551 5059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:44:08.859876 5059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:44:08.861023 5059 hybrid_clock.cc:648] HybridClock initialized: now 1786149848861000 us; error 45 us; skew 500 ppm
Aug 08 00:44:08 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149848, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:44:08.873881 5059 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:44:08.874614 5059 webserver.cc:460] Webserver started at http://127.20.105.126:46241/ using document root <none> and password file <none>
I20260808 00:44:08.874940 5059 fs_manager.cc:359] Metadata directory not provided
I20260808 00:44:08.875020 5059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:44:08.875209 5059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:44:08 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149848, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:44:08.935554 5059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "b7ee6f5a81644b409c461c145fc3b085"
format_stamp: "Formatted at 2026-08-08 00:44:08 on dist-test-slave-kvfs"
server_key: "c0ec9a8e46ffc3091476e7e50cf2610e"
server_key_iv: "e8529e31de8c50ceefc9cf778a12cec4"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:08.936134 5059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "b7ee6f5a81644b409c461c145fc3b085"
format_stamp: "Formatted at 2026-08-08 00:44:08 on dist-test-slave-kvfs"
server_key: "c0ec9a8e46ffc3091476e7e50cf2610e"
server_key_iv: "e8529e31de8c50ceefc9cf778a12cec4"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:08.938405 5059 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:44:08.954708 5082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:08.954931 5059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:44:08.955045 5059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "b7ee6f5a81644b409c461c145fc3b085"
format_stamp: "Formatted at 2026-08-08 00:44:08 on dist-test-slave-kvfs"
server_key: "c0ec9a8e46ffc3091476e7e50cf2610e"
server_key_iv: "e8529e31de8c50ceefc9cf778a12cec4"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:08.955183 5059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:44:08.977604 5059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:44:08.980320 5059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:44:08.980582 5059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:44:08.989758 5059 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:32877
I20260808 00:44:08.989801 5161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:32877 every 8 connection(s)
I20260808 00:44:08.990319 5059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:44:08.991298 5162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:44:08.994189 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 5059
I20260808 00:44:08.994306 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:44:08.994517 20901 external_mini_cluster.cc:1428] Setting key eac6b0a46cd5e9233e5ccdcf26d84b24
I20260808 00:44:08.995545 5162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085: Bootstrap starting.
I20260808 00:44:08.996641 5162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085: Neither blocks nor log segments found. Creating new log.
I20260808 00:44:08.997134 5162 log.cc:824] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085: Log is configured to *not* fsync() on all Append() calls
I20260808 00:44:08.998207 5162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085: No bootstrap required, opened a new log
I20260808 00:44:09.001556 5162 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ee6f5a81644b409c461c145fc3b085" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32877 } }
I20260808 00:44:09.001847 5162 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:44:09.001925 5162 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ee6f5a81644b409c461c145fc3b085, State: Initialized, Role: FOLLOWER
I20260808 00:44:09.002135 5162 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ee6f5a81644b409c461c145fc3b085" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32877 } }
I20260808 00:44:09.002243 5162 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:44:09.002287 5162 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:44:09.002341 5162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:44:09.003156 5162 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ee6f5a81644b409c461c145fc3b085" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32877 } }
I20260808 00:44:09.003298 5162 leader_election.cc:302] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7ee6f5a81644b409c461c145fc3b085; no voters:
I20260808 00:44:09.003556 5162 leader_election.cc:288] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:44:09.003686 5170 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:44:09.003991 5170 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [term 1 LEADER]: Becoming Leader. State: Replica: b7ee6f5a81644b409c461c145fc3b085, State: Running, Role: LEADER
I20260808 00:44:09.004118 5162 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:44:09.004185 5170 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ee6f5a81644b409c461c145fc3b085" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32877 } }
I20260808 00:44:09.004771 5170 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7ee6f5a81644b409c461c145fc3b085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ee6f5a81644b409c461c145fc3b085" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32877 } } }
I20260808 00:44:09.004980 5170 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [sys.catalog]: This master's current role is: LEADER
Aug 08 00:44:09 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:44:09.005357 5170 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ee6f5a81644b409c461c145fc3b085. Latest consensus state: current_term: 1 leader_uuid: "b7ee6f5a81644b409c461c145fc3b085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ee6f5a81644b409c461c145fc3b085" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 32877 } } }
I20260808 00:44:09.005503 5170 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085 [sys.catalog]: This master's current role is: LEADER
I20260808 00:44:09.005908 5181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:44:09.007359 5181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:44:09.008213 5162 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 00:44:09.011636 5166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:08.995382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56252 (local address 127.20.105.126:32877)
0808 00:44:08.995649 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:08.995661 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:08.995699 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:44:08.996452 (+ 753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:08.996465 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:08.996599 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:08.997512 (+ 913us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:44:08.997911 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:08.999694 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:09.000848 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:09.001685 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:09.001823 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:09.005685 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:44:09.005690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:44:09.005691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:44:09.005706 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:44:09.009291 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:09.009750 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:09.009753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:09.009754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:09.009863 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:09.010097 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:09.010097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:09.010098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:09.010439 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:44:09.010602 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:09.011228 (+ 626us) server_negotiation.cc:299] Negotiation successful
0808 00:44:09.011379 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":148,"threads_started":1}
I20260808 00:44:09.021003 5181 catalog_manager.cc:1347] Generated new cluster ID: 96afc3ace12040f1aa60a3c2ed13c90b
I20260808 00:44:09.021322 5181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:44:09.036972 5181 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:44:09.038386 5181 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:44:09.044077 5181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085: Generated new TSK 0
I20260808 00:44:09.044486 5181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:44:09.963703 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:40753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32877
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:44:10.107841 5230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:44:10.108239 5230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:44:10.108278 5230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:44:10.109890 5230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:44:10.109972 5230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:44:10.110023 5230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:44:10.111805 5230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.105.84:40753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:44:10.112138 5230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:44:10.112452 5230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:44:10.120358 5245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:44:10.120394 5242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:44:10.120404 5243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:44:10.120671 5230 server_base.cc:1034] running on GCE node
I20260808 00:44:10.120891 5230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:44:10.121173 5230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:44:10.122386 5230 hybrid_clock.cc:648] HybridClock initialized: now 1786149850122359 us; error 50 us; skew 500 ppm
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:44:10.125917 5230 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:44:10.126572 5230 webserver.cc:460] Webserver started at http://127.20.105.65:46767/ using document root <none> and password file <none>
I20260808 00:44:10.126900 5230 fs_manager.cc:359] Metadata directory not provided
I20260808 00:44:10.126973 5230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:44:10.127156 5230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:44:10.146420 5230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "c21d7dbf56af45749fa1082e17458902"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "3c292fd0dcc97e1aa739edf3bb062be8"
server_key_iv: "38c8371dfa8b75419fcd450ed394f606"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.147270 5230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "c21d7dbf56af45749fa1082e17458902"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "3c292fd0dcc97e1aa739edf3bb062be8"
server_key_iv: "38c8371dfa8b75419fcd450ed394f606"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.151467 5230 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s
I20260808 00:44:10.167768 5257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.168134 5230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:44:10.168259 5230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "c21d7dbf56af45749fa1082e17458902"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "3c292fd0dcc97e1aa739edf3bb062be8"
server_key_iv: "38c8371dfa8b75419fcd450ed394f606"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.168385 5230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:44:10.193864 5230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:44:10.196998 5230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:44:10.197335 5230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:44:10.197685 5230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:44:10.198328 5230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:44:10.198402 5230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.198493 5230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:44:10.198534 5230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.210321 5230 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:40873
I20260808 00:44:10.210971 5230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:44:10.211978 5412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:40873 every 8 connection(s)
I20260808 00:44:10.215644 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 5230
I20260808 00:44:10.215797 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:44:10.216079 20901 external_mini_cluster.cc:1428] Setting key 160305faf6e354308d13c7d9912c01c2
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:44:10.230331 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.214491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:55421 (local address 127.20.105.126:32877)
0808 00:44:10.214679 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:10.214682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:10.214972 (+ 290us) server_negotiation.cc:407] Connection header received
0808 00:44:10.216268 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:10.216270 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:10.216290 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:10.216363 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:44:10.218071 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.219030 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.220627 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.220848 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.220953 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.225414 (+ 4461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:44:10.225417 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:44:10.225417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:44:10.225430 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:44:10.227943 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:10.228694 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:10.228696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:10.228697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:10.228791 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:10.229202 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:10.229203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:10.229203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:10.229418 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:44:10.229501 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:10.229995 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 00:44:10.230138 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":99,"threads_started":1}
I20260808 00:44:10.230492 5417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.214480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:32877 (local address 127.20.105.65:55421)
0808 00:44:10.214800 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:10.214852 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:44:10.216003 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:44:10.216914 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:44:10.216921 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:44:10.217485 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:44:10.217910 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.217920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.219169 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.219171 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.220498 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.220501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.220977 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.220978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.221176 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.222308 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:44:10.222321 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:44:10.225232 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:44:10.228131 (+ 2899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:44:10.228146 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:44:10.228150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:44:10.228565 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:44:10.228877 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:44:10.228878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:44:10.228878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:44:10.229090 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:44:10.229506 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:44:10.229516 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:44:10.229762 (+ 246us) client_negotiation.cc:769] Sending connection context
0808 00:44:10.229907 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 00:44:10.230138 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":104,"threads_started":1}
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
I20260808 00:44:10.231365 5413 heartbeater.cc:347] Connected to a master server at 127.20.105.126:32877
I20260808 00:44:10.231619 5413 heartbeater.cc:464] Registering TS with master...
I20260808 00:44:10.232105 5413 heartbeater.cc:511] Master 127.20.105.126:32877 requested a full tablet report, sending...
I20260808 00:44:10.232995 5110 ts_manager.cc:194] Registered new tserver with Master: c21d7dbf56af45749fa1082e17458902 (127.20.105.65:40873)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
I20260808 00:44:10.234365 5110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:55421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:44:10.259950 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:40753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32877
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:44:10.404323 5426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:44:10.404727 5426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:44:10.404780 5426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 00:44:10.406777 5426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:44:10.406854 5426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:44:10.406909 5426 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:44:10.408996 5426 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.105.84:40753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:44:10.409330 5426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:44:10.409657 5426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:44:10.421512 5438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:44:10.421988 5436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:44:10.422452 5435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:44:10.425797 5426 server_base.cc:1034] running on GCE node
I20260808 00:44:10.426331 5426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:44:10.426736 5426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:44:10.434190 5426 hybrid_clock.cc:648] HybridClock initialized: now 1786149850434125 us; error 71 us; skew 500 ppm
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:44:10.439706 5426 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:44:10.440490 5426 webserver.cc:460] Webserver started at http://127.20.105.66:43957/ using document root <none> and password file <none>
I20260808 00:44:10.440840 5426 fs_manager.cc:359] Metadata directory not provided
I20260808 00:44:10.440910 5426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:44:10.441133 5426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:44:10.458817 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "b982ea4d14eca27bef8873e9b6122d1b"
server_key_iv: "68d486dc13e9f0cda8e4276776c74556"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.460893 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "b982ea4d14eca27bef8873e9b6122d1b"
server_key_iv: "68d486dc13e9f0cda8e4276776c74556"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.463805 5426 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:44:10.480782 5447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.481081 5426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:44:10.481220 5426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "b982ea4d14eca27bef8873e9b6122d1b"
server_key_iv: "68d486dc13e9f0cda8e4276776c74556"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.481383 5426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:44:10.500644 5426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:44:10.503785 5426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:44:10.504109 5426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:44:10.504586 5426 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:44:10.505162 5426 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:44:10.505264 5426 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.505347 5426 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:44:10.505429 5426 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.519338 5426 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:45261
I20260808 00:44:10.519536 5601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:45261 every 8 connection(s)
I20260808 00:44:10.519917 5426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:44:10.526854 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 5426
I20260808 00:44:10.527006 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:44:10.527251 20901 external_mini_cluster.cc:1428] Setting key 93a8c0673ec68851c5a259c39c380731
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:44:10.535609 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.521119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:45923 (local address 127.20.105.126:32877)
0808 00:44:10.521194 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:10.521198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:10.521574 (+ 376us) server_negotiation.cc:407] Connection header received
0808 00:44:10.522368 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:10.522370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:10.522404 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:10.522539 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:44:10.523401 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.524549 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.526231 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.526626 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.526714 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.531120 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:44:10.531122 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:44:10.531123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:44:10.531134 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:44:10.533248 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:10.534026 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:10.534028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:10.534028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:10.534110 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:10.534526 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:10.534545 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:10.534545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:10.534749 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:44:10.534806 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:10.535362 (+ 556us) server_negotiation.cc:299] Negotiation successful
0808 00:44:10.535486 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:44:10.536408 5606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.521088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:32877 (local address 127.20.105.66:45923)
0808 00:44:10.521340 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:10.521399 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 00:44:10.522215 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:44:10.522613 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:44:10.522617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:44:10.522898 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:44:10.523278 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.523283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.524693 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.524694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.526112 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.526117 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.526819 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.526820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.527341 (+ 521us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.528576 (+ 1235us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:44:10.528590 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:44:10.530939 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:44:10.533396 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:44:10.533406 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:44:10.533410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:44:10.533875 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:44:10.534201 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:44:10.534202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:44:10.534202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:44:10.534423 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:44:10.534871 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:44:10.534879 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:44:10.535127 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 00:44:10.535803 (+ 676us) client_negotiation.cc:240] Negotiation successful
0808 00:44:10.536062 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
I20260808 00:44:10.537271 5602 heartbeater.cc:347] Connected to a master server at 127.20.105.126:32877
I20260808 00:44:10.537441 5602 heartbeater.cc:464] Registering TS with master...
I20260808 00:44:10.537719 5602 heartbeater.cc:511] Master 127.20.105.126:32877 requested a full tablet report, sending...
I20260808 00:44:10.538264 5110 ts_manager.cc:194] Registered new tserver with Master: bc9ffde296bd4f9e9caf3aea6f62ba04 (127.20.105.66:45261)
I20260808 00:44:10.538841 5110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:45923
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:44:10.569901 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:40753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32877
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:44:10.759857 5613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:44:10.760253 5613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:44:10.760309 5613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:44:10.762131 5613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:44:10.762199 5613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:44:10.762247 5613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:44:10.764482 5613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.105.84:40753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:32877
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:44:10.764827 5613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:44:10.765131 5613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:44:10.775565 5622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:44:10.775754 5624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:44:10.775527 5627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:44:10.776708 5613 server_base.cc:1034] running on GCE node
I20260808 00:44:10.777004 5613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:44:10.777362 5613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:44:10.778553 5613 hybrid_clock.cc:648] HybridClock initialized: now 1786149850778509 us; error 70 us; skew 500 ppm
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:44:10.783296 5613 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:44:10.784161 5613 webserver.cc:460] Webserver started at http://127.20.105.67:34243/ using document root <none> and password file <none>
I20260808 00:44:10.784579 5613 fs_manager.cc:359] Metadata directory not provided
I20260808 00:44:10.784668 5613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:44:10.784837 5613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:44:10.806092 5613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "366c6fcdb0394f38a11261c8d0fc0fde"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "530c10c72f3be5271a0bbfd62b9700ab"
server_key_iv: "c01ce75837657b922da8bab4591ee193"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.806999 5613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "366c6fcdb0394f38a11261c8d0fc0fde"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "530c10c72f3be5271a0bbfd62b9700ab"
server_key_iv: "c01ce75837657b922da8bab4591ee193"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.816174 5613 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.002s
I20260808 00:44:10.841029 5636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.841337 5613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:44:10.841511 5613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "366c6fcdb0394f38a11261c8d0fc0fde"
format_stamp: "Formatted at 2026-08-08 00:44:10 on dist-test-slave-kvfs"
server_key: "530c10c72f3be5271a0bbfd62b9700ab"
server_key_iv: "c01ce75837657b922da8bab4591ee193"
server_key_version: "kuduclusterkey@0"
I20260808 00:44:10.841686 5613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:44:10.871023 5613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:44:10.874485 5613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:44:10.874686 5613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:44:10.875162 5613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:44:10.875751 5613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:44:10.875808 5613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.875844 5613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:44:10.875873 5613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:44:10.901129 5613 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:44779
I20260808 00:44:10.901365 5784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:44779 every 8 connection(s)
I20260808 00:44:10.902029 5613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:44:10.908198 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 5613
I20260808 00:44:10.908358 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:44:10.908581 20901 external_mini_cluster.cc:1428] Setting key 79263aed0511cf0d302195fc01bd2a81
Aug 08 00:44:10 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:44:10.916751 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.903352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:38075 (local address 127.20.105.126:32877)
0808 00:44:10.903522 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:10.903524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:10.904130 (+ 606us) server_negotiation.cc:407] Connection header received
0808 00:44:10.906165 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:10.906167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:10.906182 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:10.906237 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:44:10.907928 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.908816 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.910389 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.910613 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.910709 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.913411 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:44:10.913412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:44:10.913413 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:44:10.913422 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:44:10.914958 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:10.915475 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:10.915476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:10.915477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:10.915560 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:10.915856 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:10.915857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:10.915857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:10.916014 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:44:10.916109 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:10.916525 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 00:44:10.916630 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 00:44:10.917661 5789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.903343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:32877 (local address 127.20.105.67:38075)
0808 00:44:10.903790 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:10.903874 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0808 00:44:10.905935 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:44:10.906451 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:44:10.906459 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:44:10.907074 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:44:10.907737 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.907749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.908937 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.908938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.910284 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.910288 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.910735 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.910736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.910943 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.911621 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:44:10.911632 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:44:10.913298 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:44:10.915066 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:44:10.915075 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:44:10.915078 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:44:10.915342 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:44:10.915641 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:44:10.915642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:44:10.915643 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:44:10.915772 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:44:10.916141 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:44:10.916149 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:44:10.916344 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 00:44:10.916925 (+ 581us) client_negotiation.cc:240] Negotiation successful
0808 00:44:10.917150 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":225,"threads_started":1}
I20260808 00:44:10.918229 5786 heartbeater.cc:347] Connected to a master server at 127.20.105.126:32877
I20260808 00:44:10.918351 5786 heartbeater.cc:464] Registering TS with master...
I20260808 00:44:10.918603 5786 heartbeater.cc:511] Master 127.20.105.126:32877 requested a full tablet report, sending...
I20260808 00:44:10.919087 5110 ts_manager.cc:194] Registered new tserver with Master: 366c6fcdb0394f38a11261c8d0fc0fde (127.20.105.67:44779)
I20260808 00:44:10.919600 5110 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:38075
I20260808 00:44:10.922456 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:44:10.935268 5415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.925647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.20.105.126:32877)
0808 00:44:10.925707 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:10.925710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:10.926041 (+ 331us) server_negotiation.cc:407] Connection header received
0808 00:44:10.926163 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:10.926165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:10.926178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:10.926211 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:44:10.927606 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.928502 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.929838 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.930069 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.930166 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.931952 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:44:10.931953 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:44:10.931954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:44:10.931963 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:44:10.933733 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:10.934201 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:10.934203 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:10.934204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:10.934281 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:44:10.934538 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:44:10.934539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:44:10.934540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:44:10.934693 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:44:10.934829 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:10.935009 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 00:44:10.935121 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:44:10.937597 5110 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56260:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:44:10.941793 5110 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:44:10.961714 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.953849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:45261 (local address 127.0.0.1:43904)
0808 00:44:10.954978 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:10.955053 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0808 00:44:10.955162 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:44:10.955649 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:44:10.955653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:44:10.955656 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:44:10.955839 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.955843 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.957636 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.957637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.959170 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.959174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.961236 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.961238 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.961379 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.961401 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 00:44:10.961500 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 00:44:10.961587 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1089,"thread_start_us":213,"threads_started":1}
I20260808 00:44:10.962488 5808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.954338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44779 (local address 127.0.0.1:56332)
0808 00:44:10.955621 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:10.955633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:44:10.955699 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:44:10.956326 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:44:10.956327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:44:10.956330 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:44:10.956435 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.956438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.958570 (+ 2132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.958571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.959799 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.959803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.962052 (+ 2249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.962053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.962132 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.962138 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:44:10.962340 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 00:44:10.962398 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1261,"thread_start_us":162,"threads_started":1}
I20260808 00:44:10.962494 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.954019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43904 (local address 127.20.105.66:45261)
0808 00:44:10.954882 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:10.954890 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:10.955270 (+ 380us) server_negotiation.cc:407] Connection header received
0808 00:44:10.955333 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:10.955336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:10.955357 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:10.955432 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:44:10.955946 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.957485 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.959442 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.960898 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.960999 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.961183 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:10.962264 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0808 00:44:10.962334 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":803,"thread_start_us":200,"threads_started":1}
I20260808 00:44:10.963248 5811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.954975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56332 (local address 127.20.105.67:44779)
0808 00:44:10.955938 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:10.955944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:10.955957 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:44:10.956080 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:10.956083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:10.956107 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:10.956203 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:44:10.956540 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.958471 (+ 1931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.960127 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.961284 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.961797 (+ 513us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.961998 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:10.963034 (+ 1036us) server_negotiation.cc:299] Negotiation successful
0808 00:44:10.963093 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":856,"thread_start_us":107,"threads_started":1}
I20260808 00:44:10.963528 5505 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0458a9cb78476096aa23da239969f3 (DEFAULT_TABLE table=test-table [id=b8624db8c7a6425f80b2854cfb8466c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:44:10.964193 5505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0458a9cb78476096aa23da239969f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:44:10.964340 5698 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0458a9cb78476096aa23da239969f3 (DEFAULT_TABLE table=test-table [id=b8624db8c7a6425f80b2854cfb8466c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:44:10.964845 5698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0458a9cb78476096aa23da239969f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:44:10.966198 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.954629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40873 (local address 127.0.0.1:43986)
0808 00:44:10.955817 (+ 1188us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:10.955859 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:44:10.955962 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:44:10.956467 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:44:10.956469 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:44:10.956471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:44:10.956528 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.956529 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.958343 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.958344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.964471 (+ 6127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:10.964475 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.965897 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.965898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:10.965982 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.965999 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 00:44:10.966060 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:44:10.966112 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"thread_start_us":134,"threads_started":1}
I20260808 00:44:10.970862 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:10.954773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.20.105.65:40873)
0808 00:44:10.956177 (+ 1404us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:10.956182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:10.956194 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:44:10.956237 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:10.956239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:10.956256 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:10.956340 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:44:10.956635 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.958232 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.964637 (+ 6405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:10.965702 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:10.965786 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:10.965849 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:10.970668 (+ 4819us) server_negotiation.cc:299] Negotiation successful
0808 00:44:10.970746 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1349,"thread_start_us":73,"threads_started":1}
I20260808 00:44:10.971626 5812 tablet_bootstrap.cc:492] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Bootstrap starting.
I20260808 00:44:10.971808 5327 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0458a9cb78476096aa23da239969f3 (DEFAULT_TABLE table=test-table [id=b8624db8c7a6425f80b2854cfb8466c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:44:10.972390 5327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0458a9cb78476096aa23da239969f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:44:10.974270 5812 tablet_bootstrap.cc:654] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Neither blocks nor log segments found. Creating new log.
I20260808 00:44:10.975078 5812 log.cc:824] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Log is configured to *not* fsync() on all Append() calls
I20260808 00:44:10.975788 5814 tablet_bootstrap.cc:492] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Bootstrap starting.
I20260808 00:44:10.976527 5812 tablet_bootstrap.cc:492] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: No bootstrap required, opened a new log
I20260808 00:44:10.976719 5812 ts_tablet_manager.cc:1397] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 00:44:10.979367 5816 tablet_bootstrap.cc:492] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902: Bootstrap starting.
I20260808 00:44:10.982046 5816 tablet_bootstrap.cc:654] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902: Neither blocks nor log segments found. Creating new log.
I20260808 00:44:10.982805 5814 tablet_bootstrap.cc:654] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Neither blocks nor log segments found. Creating new log.
I20260808 00:44:10.982853 5816 log.cc:824] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902: Log is configured to *not* fsync() on all Append() calls
I20260808 00:44:10.983796 5812 raft_consensus.cc:348] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:10.984237 5812 raft_consensus.cc:374] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:44:10.984388 5816 tablet_bootstrap.cc:492] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902: No bootstrap required, opened a new log
I20260808 00:44:10.984478 5816 ts_tablet_manager.cc:1397] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260808 00:44:10.985037 5812 raft_consensus.cc:729] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc9ffde296bd4f9e9caf3aea6f62ba04, State: Initialized, Role: FOLLOWER
I20260808 00:44:10.985311 5812 consensus_queue.cc:260] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:10.985698 5812 ts_tablet_manager.cc:1428] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260808 00:44:10.985894 5602 heartbeater.cc:503] Master 127.20.105.126:32877 was elected leader, sending a full tablet report...
I20260808 00:44:10.985975 5814 log.cc:824] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Log is configured to *not* fsync() on all Append() calls
I20260808 00:44:10.989975 5814 tablet_bootstrap.cc:492] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: No bootstrap required, opened a new log
I20260808 00:44:10.990113 5814 ts_tablet_manager.cc:1397] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260808 00:44:10.990794 5816 raft_consensus.cc:348] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:10.991235 5816 raft_consensus.cc:374] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:44:10.991386 5816 raft_consensus.cc:729] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c21d7dbf56af45749fa1082e17458902, State: Initialized, Role: FOLLOWER
I20260808 00:44:10.991706 5816 consensus_queue.cc:260] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:10.992311 5816 ts_tablet_manager.cc:1428] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20260808 00:44:10.992575 5413 heartbeater.cc:503] Master 127.20.105.126:32877 was elected leader, sending a full tablet report...
I20260808 00:44:10.995798 5814 raft_consensus.cc:348] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:10.996068 5814 raft_consensus.cc:374] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:44:10.996923 5814 raft_consensus.cc:729] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 366c6fcdb0394f38a11261c8d0fc0fde, State: Initialized, Role: FOLLOWER
I20260808 00:44:10.997243 5814 consensus_queue.cc:260] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:10.997777 5814 ts_tablet_manager.cc:1428] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Time spent starting tablet: real 0.008s user 0.001s sys 0.005s
I20260808 00:44:10.998090 5786 heartbeater.cc:503] Master 127.20.105.126:32877 was elected leader, sending a full tablet report...
W20260808 00:44:11.021140 5605 tablet.cc:2369] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 00:44:11 dist-test-slave-kvfs krb5kdc[4416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149850, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:44:11.135193 5822 raft_consensus.cc:482] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:44:11.135488 5822 raft_consensus.cc:504] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:11.136178 5822 leader_election.cc:288] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc9ffde296bd4f9e9caf3aea6f62ba04 (127.20.105.66:45261), 366c6fcdb0394f38a11261c8d0fc0fde (127.20.105.67:44779)
I20260808 00:44:11.146936 5806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:11.137077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:34373 (local address 127.20.105.66:45261)
0808 00:44:11.137147 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:11.137150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:11.141854 (+ 4704us) server_negotiation.cc:407] Connection header received
0808 00:44:11.142014 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:11.142016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:11.142031 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:11.142068 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:44:11.142752 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.143532 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.145518 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.146378 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.146436 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:11.146450 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:11.146768 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 00:44:11.146831 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:44:11.147316 5825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:11.136388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:45261 (local address 127.20.105.65:34373)
0808 00:44:11.141739 (+ 5351us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:11.141757 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:44:11.141900 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:44:11.142170 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:44:11.142171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:44:11.142192 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:44:11.142615 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:11.142620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.143671 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.143672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:11.145384 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:11.145389 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.146518 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.146520 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:11.146656 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:11.146662 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:44:11.147130 (+ 468us) client_negotiation.cc:240] Negotiation successful
0808 00:44:11.147208 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5272,"thread_start_us":205,"threads_started":1}
I20260808 00:44:11.148881 5530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0458a9cb78476096aa23da239969f3" candidate_uuid: "c21d7dbf56af45749fa1082e17458902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" is_pre_election: true
I20260808 00:44:11.149192 5530 raft_consensus.cc:2454] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c21d7dbf56af45749fa1082e17458902 in term 0.
I20260808 00:44:11.149569 5260 leader_election.cc:302] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc9ffde296bd4f9e9caf3aea6f62ba04, c21d7dbf56af45749fa1082e17458902; no voters:
I20260808 00:44:11.149782 5822 raft_consensus.cc:2781] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:44:11.149859 5822 raft_consensus.cc:482] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:44:11.149896 5822 raft_consensus.cc:3037] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:44:11.151373 5822 raft_consensus.cc:504] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:11.151595 5822 leader_election.cc:288] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [CANDIDATE]: Term 1 election: Requested vote from peers bc9ffde296bd4f9e9caf3aea6f62ba04 (127.20.105.66:45261), 366c6fcdb0394f38a11261c8d0fc0fde (127.20.105.67:44779)
I20260808 00:44:11.151895 5530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0458a9cb78476096aa23da239969f3" candidate_uuid: "c21d7dbf56af45749fa1082e17458902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04"
I20260808 00:44:11.152030 5530 raft_consensus.cc:3037] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:44:11.153476 5530 raft_consensus.cc:2454] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c21d7dbf56af45749fa1082e17458902 in term 1.
W20260808 00:44:11.153825 5787 tablet.cc:2369] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:44:11.154253 5260 leader_election.cc:302] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc9ffde296bd4f9e9caf3aea6f62ba04, c21d7dbf56af45749fa1082e17458902; no voters:
I20260808 00:44:11.154376 5822 raft_consensus.cc:2781] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:44:11.154598 5822 raft_consensus.cc:686] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [term 1 LEADER]: Becoming Leader. State: Replica: c21d7dbf56af45749fa1082e17458902, State: Running, Role: LEADER
I20260808 00:44:11.154721 5822 consensus_queue.cc:237] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } }
I20260808 00:44:11.156073 5811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:11.136895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:39715 (local address 127.20.105.67:44779)
0808 00:44:11.136978 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:11.136982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:11.145721 (+ 8739us) server_negotiation.cc:407] Connection header received
0808 00:44:11.145852 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:11.145853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:11.145868 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:11.145903 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:44:11.147607 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.148292 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.152536 (+ 4244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.153321 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.153389 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:11.153404 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:11.155879 (+ 2475us) server_negotiation.cc:299] Negotiation successful
0808 00:44:11.155956 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:44:11.155999 5110 catalog_manager.cc:5495] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 reported cstate change: term changed from 0 to 1, leader changed from <none> to c21d7dbf56af45749fa1082e17458902 (127.20.105.65). New cstate: current_term: 1 leader_uuid: "c21d7dbf56af45749fa1082e17458902" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c21d7dbf56af45749fa1082e17458902" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40873 } health_report { overall_health: HEALTHY } } }
I20260808 00:44:11.156621 5824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:11.136779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:44779 (local address 127.20.105.65:39715)
0808 00:44:11.145630 (+ 8851us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:11.145645 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:44:11.145762 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:44:11.145984 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:44:11.145985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:44:11.145987 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:44:11.146233 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:11.146236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.148652 (+ 2416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.148653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:11.152384 (+ 3731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:44:11.152388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.155667 (+ 3279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.155669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:44:11.155763 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:11.155768 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:44:11.156202 (+ 434us) client_negotiation.cc:240] Negotiation successful
0808 00:44:11.156266 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8823,"thread_start_us":161,"threads_started":1}
I20260808 00:44:11.157274 5728 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0458a9cb78476096aa23da239969f3" candidate_uuid: "c21d7dbf56af45749fa1082e17458902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" is_pre_election: true
I20260808 00:44:11.157476 5728 raft_consensus.cc:2454] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c21d7dbf56af45749fa1082e17458902 in term 0.
I20260808 00:44:11.157702 5727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0458a9cb78476096aa23da239969f3" candidate_uuid: "c21d7dbf56af45749fa1082e17458902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "366c6fcdb0394f38a11261c8d0fc0fde"
I20260808 00:44:11.157796 5727 raft_consensus.cc:3037] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:44:11.159638 5727 raft_consensus.cc:2454] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c21d7dbf56af45749fa1082e17458902 in term 1.
I20260808 00:44:11.191978 5810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:11.188238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44000 (local address 127.20.105.65:40873)
0808 00:44:11.188295 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:44:11.188297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:44:11.188306 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:44:11.188362 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:44:11.188363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:44:11.188391 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:44:11.188461 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:44:11.188808 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.189590 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.190804 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:44:11.191126 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:44:11.191202 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:44:11.191423 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:44:11.191602 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:44:11.191696 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 00:44:11.191753 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 00:44:11.191782 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:44:11.195082 5727 raft_consensus.cc:1261] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 1 FOLLOWER]: Refusing update from remote peer c21d7dbf56af45749fa1082e17458902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:44:11.195096 5530 raft_consensus.cc:1261] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 1 FOLLOWER]: Refusing update from remote peer c21d7dbf56af45749fa1082e17458902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:44:11.195492 5822 consensus_queue.cc:1035] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "366c6fcdb0394f38a11261c8d0fc0fde" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 44779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:44:11.195719 5822 consensus_queue.cc:1035] T 3e0458a9cb78476096aa23da239969f3 P c21d7dbf56af45749fa1082e17458902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc9ffde296bd4f9e9caf3aea6f62ba04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 45261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:44:11.201161 5831 mvcc.cc:205] Tried to move back new op lower bound from 7316069790491332608 to 7316069790330585088. Current Snapshot: MvccSnapshot[applied={T|T < 7316069790491332608}]
I20260808 00:44:11.201422 5830 mvcc.cc:205] Tried to move back new op lower bound from 7316069790491332608 to 7316069790330585088. Current Snapshot: MvccSnapshot[applied={T|T < 7316069790491332608}]
I20260808 00:44:11.207247 5110 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:44:11.207422 5110 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56260:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:44:11.208544 5110 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7ee6f5a81644b409c461c145fc3b085: Sending DeleteTablet for 3 replicas of tablet 3e0458a9cb78476096aa23da239969f3
I20260808 00:44:11.209177 5505 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0458a9cb78476096aa23da239969f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:44:11 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:43904
I20260808 00:44:11.209407 5698 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0458a9cb78476096aa23da239969f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:44:11 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:56332
I20260808 00:44:11.209733 5840 tablet_replica.cc:330] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: stopping tablet replica
I20260808 00:44:11.209816 5327 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0458a9cb78476096aa23da239969f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:44:11 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:43986
I20260808 00:44:11.209970 5840 raft_consensus.cc:2229] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:44:11.210186 5840 raft_consensus.cc:2258] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:44:11.210625 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 5230
I20260808 00:44:11.210871 5840 ts_tablet_manager.cc:1905] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:44:11.213336 5840 ts_tablet_manager.cc:1918] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:44:11.213475 5840 log.cc:1196] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/3e0458a9cb78476096aa23da239969f3
I20260808 00:44:11.213793 5840 ts_tablet_manager.cc:1939] T 3e0458a9cb78476096aa23da239969f3 P bc9ffde296bd4f9e9caf3aea6f62ba04: Deleting consensus metadata
I20260808 00:44:11.214828 5087 catalog_manager.cc:4841] TS bc9ffde296bd4f9e9caf3aea6f62ba04 (127.20.105.66:45261): tablet 3e0458a9cb78476096aa23da239969f3 (table test-table [id=b8624db8c7a6425f80b2854cfb8466c1]) successfully deleted
I20260808 00:44:11.220346 5841 tablet_replica.cc:330] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: stopping tablet replica
I20260808 00:44:11.220846 5841 raft_consensus.cc:2229] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:44:11.221145 5841 raft_consensus.cc:2258] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:44:11.221760 5841 ts_tablet_manager.cc:1905] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:44:11.224134 5841 ts_tablet_manager.cc:1918] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:44:11.224283 5841 log.cc:1196] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/3e0458a9cb78476096aa23da239969f3
I20260808 00:44:11.224892 5841 ts_tablet_manager.cc:1939] T 3e0458a9cb78476096aa23da239969f3 P 366c6fcdb0394f38a11261c8d0fc0fde: Deleting consensus metadata
I20260808 00:44:11.225888 5083 catalog_manager.cc:4841] TS 366c6fcdb0394f38a11261c8d0fc0fde (127.20.105.67:44779): tablet 3e0458a9cb78476096aa23da239969f3 (table test-table [id=b8624db8c7a6425f80b2854cfb8466c1]) successfully deleted
W20260808 00:44:11.227682 5083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:40873 (error 108)
I20260808 00:44:11.228147 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 5426
I20260808 00:44:11.238008 5807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:44:11.228402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40873 (local address 127.0.0.1:44006)
0808 00:44:11.237824 (+ 9422us) negotiation.cc:107] Waiting for socket to connect
0808 00:44:11.237932 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:40873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":9379}
W20260808 00:44:11.279644 5083 catalog_manager.cc:4584] TS c21d7dbf56af45749fa1082e17458902 (127.20.105.65:40873): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e0458a9cb78476096aa23da239969f3: Network error: Client connection negotiation failed: client connection to 127.20.105.65:40873: connect: Connection refused (error 111)
I20260808 00:44:11.288110 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 5613
I20260808 00:44:11.306834 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 5059
I20260808 00:44:11.333989 20901 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 00:44:11.689597 20901 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 00:44:11.689781 20901 mini_ranger.cc:66] Stopping Ranger...
I20260808 00:44:12.045760 20901 mini_ranger.cc:68] Stopped Ranger
2026-08-08 00:44:12.046 UTC [4456] LOG: received smart shutdown request
2026-08-08 00:44:12.051 UTC [4456] LOG: background worker "logical replication launcher" (PID 4467) exited with exit code 1
2026-08-08 00:44:12.051 UTC [4461] LOG: shutting down
2026-08-08 00:44:12.248 UTC [4456] LOG: database system is shut down
2026-08-08T00:44:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (56038 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:44:12 dist-test-slave-kvfs krb5kdc[5855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:44:12 dist-test-slave-kvfs krb5kdc[5855](info): set up 2 sockets
Aug 08 00:44:12 dist-test-slave-kvfs krb5kdc[5855](info): commencing operation
krb5kdc: starting...
W20260808 00:44:13.659890 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:44:13 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:44:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/rangeradmin_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/rangeradmin_127.20.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/rangerlookup_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/rangerlookup_127.20.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/HTTP_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/HTTP_127.20.105.84@KRBTEST.COM.keytab.
I20260808 00:44:13.883957 20901 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskzO4qSW/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/postgres -l logfile start
2026-08-08 00:44:16.882 UTC [5906] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 00:44:16.882 UTC [5906] LOG: listening on IPv4 address "127.20.105.84", port 45299
2026-08-08 00:44:16.885 UTC [5906] LOG: listening on Unix socket "/tmp/.s.PGSQL.45299"
2026-08-08 00:44:16.911 UTC [5910] LOG: database system was shut down at 2026-08-08 00:44:15 UTC
2026-08-08 00:44:16.916 UTC [5906] LOG: database system is ready to accept connections
I20260808 00:44:18.100927 20901 mini_postgres.cc:96] Postgres bound to 45299
2026-08-08 00:44:18.110 UTC [5920] FATAL: database "slave" does not exist
127.20.105.84:45299 - accepting connections
I20260808 00:44:18.111348 20901 mini_ranger.cc:161] Starting Ranger...
I20260808 00:44:18.130352 20901 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 00:44:18.725247 20901 mini_ranger.cc:87] Created ranger Postgres database
I20260808 00:44:18.725384 20901 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-admin
2026-08-08 00:44:19,060 [I] DB FLAVOR :POSTGRES
2026-08-08 00:44:19,061 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:44:19,061 [I] Checking connection..
2026-08-08 00:44:19,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 00:44:19,348 [I] Checking connection passed.
2026-08-08 00:44:19,349 [I] --------- Verifying version history table ---------
2026-08-08 00:44:19,349 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:44:19,643 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:44:19,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:44:19,927 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:44:19,927 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 00:44:19,927 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 00:44:20.212 UTC [6012] WARNING: there is no transaction in progress
2026-08-08 00:44:20,221 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 00:44:20,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:44:20,532 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 00:44:20,533 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:44:20,533 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 00:44:20,842 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 00:44:21,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-kvfs.c.gcp-upstream.internal','N') ;"
2026-08-08 00:44:21,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:44:21,713 [I] Table x_portal_user does not exist in database ranger
2026-08-08 00:44:21,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:44:22,009 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 00:44:22,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:44:22,292 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:44:22,590 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 00:44:22,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 00:44:23.727 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.748 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.773 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.795 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.825 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.853 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.930 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.939 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.960 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.972 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:23.984 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:24.000 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:24.012 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:24.019 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:24.027 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:24.036 UTC [6174] WARNING: there is no transaction in progress
2026-08-08 00:44:24,876 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 00:44:24,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 00:44:25,140 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 00:44:25,141 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:44:25,430 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 00:44:25,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:44:25,729 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 00:44:25,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:44:26,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:44:26,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-kvfs.c.gcp-upstream.internal';"
2026-08-08 00:44:26,605 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 00:44:26,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 00:44:26,893 [I] DB_PATCHES have already been applied
I20260808 00:44:26.900279 20901 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-admin:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskzO4qSW/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 00:44:26.900393 20901 mini_ranger.cc:193] Using host: 127.20.105.84
I20260808 00:44:26.906035 20901 mini_ranger.cc:227] Ranger admin URL: http://127.20.105.84:52429
Aug 08, 2026 12:44:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:44:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:44:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 12:44:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 12:44:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 12:44:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 12:44:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 00:44:28.316998 20901 mini_ranger.cc:160] Time spent starting Ranger: real 10.206s user 0.000s sys 0.015s
Aug 08, 2026 12:44:52 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 12:44:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 12:44:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 12:44:52 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 12:44:53 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 00:44:54.110627 20901 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.105.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.105.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/rangerkms_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.105.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/rangerkms_127.20.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.105.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.105.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.105.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/HTTP_127.20.105.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.105.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/HTTP_127.20.105.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 00:44:54 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149894, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 00:44:54.230137 20901 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 00:44:54.247040 20901 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 00:44:55.347437 20901 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 00:44:55.347558 20901 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms
I20260808 00:44:55.347595 20901 mini_ranger_kms.cc:227] Using postgres at 127.20.105.84:45299
2026-08-08 00:44:55,581 [I] DB FLAVOR :POSTGRES
2026-08-08 00:44:55,581 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:44:55,581 [I] Checking connection
2026-08-08 00:44:55,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 00:44:55,860 [I] connection success
2026-08-08 00:44:55,860 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 00:44:55,860 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 00:44:55,861 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 00:44:56,143 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 00:44:56,144 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:44:56,144 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 00:44:56,144 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.105.84:45299/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 00:44:56,455 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 00:44:57.036425 20901 mini_ranger_kms.cc:310] Created kms service
I20260808 00:44:57.248087 20901 mini_ranger_kms.cc:326] Created kudu user
I20260808 00:44:57.308826 20901 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 00:44:57 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149894, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:44:57.782814 20901 mini_ranger_kms.cc:384] Added ranger policy
I20260808 00:44:57.783098 20901 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms:/tmp/dist-test-taskzO4qSW/build/release/bin/postgresql.jar:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskzO4qSW/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 00:44:57.783140 20901 mini_ranger_kms.cc:240] Using host: 127.20.105.84
I20260808 00:44:57.789108 20901 mini_ranger_kms.cc:276] Ranger KMS PID: 6441
I20260808 00:44:57.789219 20901 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.105.84:55765
Aug 08, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:44:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:44:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 12:44:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 12:44:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 12:44:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 00:45:00 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149900, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.105.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:00.702980 20901 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.473s user 0.000s sys 0.217s
I20260808 00:45:00.703135 20901 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 00:45:00.703198 20901 mini_ranger_kms.cc:401] 127.20.105.84:55765/kms/v1/keys
Aug 08 00:45:00 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149900, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.105.84@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
Aug 08 00:45:01 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149900, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.105.84@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
Aug 08 00:45:03 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:45:04.032563 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:39271
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:39271
--ranger_config_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:55765/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:04.139292 6508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:04.139559 6508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:04.139585 6508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:04.141003 6508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:04.141062 6508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:04.141076 6508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:04.141089 6508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:04.141103 6508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:04.143004 6508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.105.84:55765/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:39271
--ranger_config_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:39271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:04.143321 6508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:04.143647 6508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:04.150246 6518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:04.150247 6516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:04.150383 6508 server_base.cc:1034] running on GCE node
W20260808 00:45:04.150290 6523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:04.151127 6508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:04.151643 6508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:04.152848 6508 hybrid_clock.cc:648] HybridClock initialized: now 1786149904152816 us; error 47 us; skew 500 ppm
Aug 08 00:45:04 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149904, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:04.156977 6508 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:04.157629 6508 webserver.cc:460] Webserver started at http://127.20.105.126:39479/ using document root <none> and password file <none>
I20260808 00:45:04.157935 6508 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:04.158012 6508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:04.158195 6508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:45:04 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149904, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:04.238958 6508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "2707680cff2842068b20eeb7895e41f1"
format_stamp: "Formatted at 2026-08-08 00:45:04 on dist-test-slave-kvfs"
server_key: "f1b0b8ba24404eb4cd0629482f2cf172"
server_key_iv: "d82dfb1debbcd4d9e0f36d15884909f2"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:04.239614 6508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "2707680cff2842068b20eeb7895e41f1"
format_stamp: "Formatted at 2026-08-08 00:45:04 on dist-test-slave-kvfs"
server_key: "f1b0b8ba24404eb4cd0629482f2cf172"
server_key_iv: "d82dfb1debbcd4d9e0f36d15884909f2"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:04.241539 6508 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 00:45:04.262621 6533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:04.262917 6508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:45:04.263069 6508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "2707680cff2842068b20eeb7895e41f1"
format_stamp: "Formatted at 2026-08-08 00:45:04 on dist-test-slave-kvfs"
server_key: "f1b0b8ba24404eb4cd0629482f2cf172"
server_key_iv: "d82dfb1debbcd4d9e0f36d15884909f2"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:04.263217 6508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:04.275672 6508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:04.278436 6508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:04.278647 6508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:04.285322 6508 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:39271
I20260808 00:45:04.285414 6606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:39271 every 8 connection(s)
I20260808 00:45:04.285946 6508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:04.286908 6607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:04.290208 6607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Bootstrap starting.
I20260808 00:45:04.291038 6607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:04.291435 6607 log.cc:824] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:04.292472 6607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: No bootstrap required, opened a new log
I20260808 00:45:04.294533 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 6508
I20260808 00:45:04.294691 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:45:04.294960 20901 external_mini_cluster.cc:1428] Setting key db9a92900e6a649ee72c03620506db58
I20260808 00:45:04.295590 6607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } }
I20260808 00:45:04.295902 6607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:04.296010 6607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2707680cff2842068b20eeb7895e41f1, State: Initialized, Role: FOLLOWER
I20260808 00:45:04.296298 6607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } }
I20260808 00:45:04.296429 6607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:45:04.296496 6607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:45:04.296567 6607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:04.297681 6607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } }
I20260808 00:45:04.297845 6607 leader_election.cc:302] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2707680cff2842068b20eeb7895e41f1; no voters:
I20260808 00:45:04.298097 6607 leader_election.cc:288] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:45:04.298132 6613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:04.298420 6613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 1 LEADER]: Becoming Leader. State: Replica: 2707680cff2842068b20eeb7895e41f1, State: Running, Role: LEADER
I20260808 00:45:04.298520 6607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:04.298621 6613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } }
I20260808 00:45:04.299098 6613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2707680cff2842068b20eeb7895e41f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } } }
I20260808 00:45:04.299299 6613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:04.299584 6613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2707680cff2842068b20eeb7895e41f1. Latest consensus state: current_term: 1 leader_uuid: "2707680cff2842068b20eeb7895e41f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } } }
I20260808 00:45:04.299714 6613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:04.299945 6627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:04.300442 6607 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 00:45:04.300747 6627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:45:04 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:04.304224 6637 mvcc.cc:205] Tried to move back new op lower bound from 7316070008022106112 to 7316070008007802880. Current Snapshot: MvccSnapshot[applied={T|T < 7316070008022106112}]
I20260808 00:45:04.304998 6627 catalog_manager.cc:1347] Generated new cluster ID: 7775c2af25d046c5a06abb869f870c3f
I20260808 00:45:04.305125 6627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:45:04.309450 6611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:04.295729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47214 (local address 127.20.105.126:39271)
0808 00:45:04.295990 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:04.296004 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:04.296074 (+ 70us) server_negotiation.cc:407] Connection header received
0808 00:45:04.296365 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:04.296376 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:04.296452 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:04.296637 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:04.297698 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:04.299288 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:04.300491 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:04.301160 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:04.301281 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:04.304714 (+ 3433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:04.304719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:04.304719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:04.304735 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:04.307035 (+ 2300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:04.307492 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:04.307496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:04.307496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:04.307649 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:04.307855 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:04.307856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:04.307857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:04.308259 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:04.308957 (+ 698us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:04.309047 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 00:45:04.309196 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":131,"threads_started":1}
I20260808 00:45:04.335016 6627 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:45:04.336097 6627 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:45:04.349582 6627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Generated new TSK 0
I20260808 00:45:04.350052 6627 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:45:05.280169 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:55765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:39271
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:05.394121 6667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:05.394466 6667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:05.394519 6667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:05.396214 6667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:05.396299 6667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:05.396355 6667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:45:05.398453 6667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.105.84:55765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:05.398828 6667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:05.399154 6667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:05.406144 6677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:05.406148 6673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:05.406661 6672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:05.407245 6667 server_base.cc:1034] running on GCE node
I20260808 00:45:05.407764 6667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:05.408195 6667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:05.409412 6667 hybrid_clock.cc:648] HybridClock initialized: now 1786149905409385 us; error 49 us; skew 500 ppm
Aug 08 00:45:05 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149905, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:05.413403 6667 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:45:05.414299 6667 webserver.cc:460] Webserver started at http://127.20.105.65:45103/ using document root <none> and password file <none>
I20260808 00:45:05.414701 6667 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:05.414800 6667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:05.414997 6667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:45:05 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149905, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:05.441372 6667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "cde0511636fc45b7b6a229060ac632b3"
format_stamp: "Formatted at 2026-08-08 00:45:05 on dist-test-slave-kvfs"
server_key: "a82628d54d7fcafdcef63227bbbf4887"
server_key_iv: "0ed498a7fc22e897caf200896ee46ced"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:05.442654 6667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "cde0511636fc45b7b6a229060ac632b3"
format_stamp: "Formatted at 2026-08-08 00:45:05 on dist-test-slave-kvfs"
server_key: "a82628d54d7fcafdcef63227bbbf4887"
server_key_iv: "0ed498a7fc22e897caf200896ee46ced"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:05.448740 6667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260808 00:45:05.468233 6689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:05.468467 6667 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:05.468608 6667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "cde0511636fc45b7b6a229060ac632b3"
format_stamp: "Formatted at 2026-08-08 00:45:05 on dist-test-slave-kvfs"
server_key: "a82628d54d7fcafdcef63227bbbf4887"
server_key_iv: "0ed498a7fc22e897caf200896ee46ced"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:05.468762 6667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:05.501323 6667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:05.504539 6667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:05.504758 6667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:05.505115 6667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:05.505618 6667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:05.505677 6667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:05.505782 6667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:05.505821 6667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:05.515857 6667 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:34119
I20260808 00:45:05.515988 6834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:34119 every 8 connection(s)
I20260808 00:45:05.516503 6667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:45:05 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149905, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:05.526234 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 6667
I20260808 00:45:05.526367 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:45:05.526633 20901 external_mini_cluster.cc:1428] Setting key 820c02ff6755e0d7e4dc180d919562ad
I20260808 00:45:05.530604 6842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:05.517596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:39271 (local address 127.20.105.65:52139)
0808 00:45:05.517827 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:05.517883 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:05.518695 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:05.519096 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:05.519101 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:05.519423 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:05.519699 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:05.519705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:05.520936 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:05.520937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:05.521850 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:05.521852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:05.522523 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:05.522524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:05.522688 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:05.523364 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:05.523374 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:05.525285 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:05.528346 (+ 3061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:05.528357 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:05.528361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:05.528791 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:05.529067 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:05.529068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:05.529068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:05.529264 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:05.529630 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:05.529638 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:05.529888 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 00:45:05.530034 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 00:45:05.530262 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":128,"threads_started":1}
I20260808 00:45:05.531216 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:05.517604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:52139 (local address 127.20.105.126:39271)
0808 00:45:05.517771 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:05.517774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:05.518007 (+ 233us) server_negotiation.cc:407] Connection header received
0808 00:45:05.518862 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:05.518864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:05.518886 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:05.518989 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:05.519826 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:05.520818 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:05.521951 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:05.522339 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:05.522433 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:05.525415 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:05.525417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:05.525417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:05.525429 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:05.528242 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:05.528890 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:05.528891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:05.528892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:05.528981 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:05.529350 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:05.529352 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:05.529352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:05.529539 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:05.529624 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:05.530952 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0808 00:45:05.531075 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260808 00:45:05.531685 6839 heartbeater.cc:347] Connected to a master server at 127.20.105.126:39271
I20260808 00:45:05.531831 6839 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:05.532131 6839 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:05.532845 6555 ts_manager.cc:194] Registered new tserver with Master: cde0511636fc45b7b6a229060ac632b3 (127.20.105.65:34119)
I20260808 00:45:05.533804 6555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:52139
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:45:05.572759 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:55765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:39271
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:45:05 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149905, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 00:45:05.730195 6845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:05.730659 6845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:05.730707 6845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:05.732903 6845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:05.732978 6845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:05.733028 6845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:45:05.735728 6845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.105.84:55765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:05.736125 6845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:05.736478 6845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:05.743270 6852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:05.743325 6853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:05.743446 6855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:05.743942 6845 server_base.cc:1034] running on GCE node
I20260808 00:45:05.744256 6845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:05.744642 6845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:05.745844 6845 hybrid_clock.cc:648] HybridClock initialized: now 1786149905745814 us; error 46 us; skew 500 ppm
Aug 08 00:45:05 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149905, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:05.749372 6845 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:45:05.750108 6845 webserver.cc:460] Webserver started at http://127.20.105.66:37045/ using document root <none> and password file <none>
I20260808 00:45:05.750466 6845 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:05.750548 6845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:05.750787 6845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:45:05 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149905, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:05.771422 6845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "af95958407c64107ad5aedba19bc8036"
format_stamp: "Formatted at 2026-08-08 00:45:05 on dist-test-slave-kvfs"
server_key: "26a25f1722163c6e1663306e341f0dce"
server_key_iv: "029bd31078e52d4a8380497fa5a0deee"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:05.771981 6845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "af95958407c64107ad5aedba19bc8036"
format_stamp: "Formatted at 2026-08-08 00:45:05 on dist-test-slave-kvfs"
server_key: "26a25f1722163c6e1663306e341f0dce"
server_key_iv: "029bd31078e52d4a8380497fa5a0deee"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:05.773782 6845 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 00:45:05.795266 6864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:05.795596 6845 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260808 00:45:05.795742 6845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "af95958407c64107ad5aedba19bc8036"
format_stamp: "Formatted at 2026-08-08 00:45:05 on dist-test-slave-kvfs"
server_key: "26a25f1722163c6e1663306e341f0dce"
server_key_iv: "029bd31078e52d4a8380497fa5a0deee"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:05.795885 6845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:05.817595 6845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:05.820510 6845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:05.820736 6845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:05.821058 6845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:05.821492 6845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:05.821557 6845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:05.821627 6845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:05.821681 6845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:05.831972 6845 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:34397
I20260808 00:45:05.832777 6845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:45:05.833576 7015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:34397 every 8 connection(s)
I20260808 00:45:05.839152 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 6845
I20260808 00:45:05.839303 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:45:05.839560 20901 external_mini_cluster.cc:1428] Setting key 0c88753d083c16443c491a441e3527e4
Aug 08 00:45:05 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149905, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:05.847733 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:05.834418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:49955 (local address 127.20.105.126:39271)
0808 00:45:05.834492 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:05.834495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:05.834510 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:45:05.835335 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:05.835338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:05.835361 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:05.835449 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:05.836441 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:05.837330 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:05.839508 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:05.839941 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:05.840009 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:05.843622 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:05.843624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:05.843625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:05.843636 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:05.845650 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:05.846273 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:05.846274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:05.846275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:05.846345 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:05.846704 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:05.846706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:05.846706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:05.846921 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:05.847013 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:05.847458 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 00:45:05.847608 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:45:05.848225 7020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:05.834010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:39271 (local address 127.20.105.66:49955)
0808 00:45:05.834266 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:05.834329 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:05.835177 (+ 848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:05.835544 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:05.835550 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:05.835900 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:05.836306 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:05.836314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:05.837470 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:05.837471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:05.839361 (+ 1890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:05.839366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:05.840318 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:05.840319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:05.840526 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:05.841340 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:05.841355 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:05.843449 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:05.845777 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:05.845790 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:05.845794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:05.846174 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:05.846418 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:05.846418 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:05.846419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:05.846598 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:05.847022 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:05.847032 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:05.847259 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 00:45:05.847400 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 00:45:05.847656 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":126,"threads_started":1}
I20260808 00:45:05.849169 7017 heartbeater.cc:347] Connected to a master server at 127.20.105.126:39271
I20260808 00:45:05.849308 7017 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:05.849560 7017 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:05.850102 6555 ts_manager.cc:194] Registered new tserver with Master: af95958407c64107ad5aedba19bc8036 (127.20.105.66:34397)
I20260808 00:45:05.850674 6555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:49955
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:45:05.884555 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:55765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:39271
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:06.017032 7025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:06.017506 7025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:06.017632 7025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:06.019611 7025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:06.019731 7025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:06.019825 7025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:45:06.022328 7025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.105.84:55765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:06.023010 7025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:06.023418 7025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:06.032873 7030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:06.032995 7025 server_base.cc:1034] running on GCE node
W20260808 00:45:06.033179 7033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:06.033420 7031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:06.033871 7025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:06.034266 7025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:06.035471 7025 hybrid_clock.cc:648] HybridClock initialized: now 1786149906035443 us; error 48 us; skew 500 ppm
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:06.044360 7025 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:45:06.045157 7025 webserver.cc:460] Webserver started at http://127.20.105.67:33815/ using document root <none> and password file <none>
I20260808 00:45:06.045501 7025 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:06.045588 7025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:06.045761 7025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:06.073902 7025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "4fe81f3b80d342e3939db52e5592df1d"
format_stamp: "Formatted at 2026-08-08 00:45:06 on dist-test-slave-kvfs"
server_key: "3f42c6d6e127af44c4182af7e7d8bc43"
server_key_iv: "ad6078a87f831cfbf82e48b3e7fce442"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:06.074651 7025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "4fe81f3b80d342e3939db52e5592df1d"
format_stamp: "Formatted at 2026-08-08 00:45:06 on dist-test-slave-kvfs"
server_key: "3f42c6d6e127af44c4182af7e7d8bc43"
server_key_iv: "ad6078a87f831cfbf82e48b3e7fce442"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:06.086194 7025 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.001s
I20260808 00:45:06.109920 7046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:06.110420 7025 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260808 00:45:06.110589 7025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "4fe81f3b80d342e3939db52e5592df1d"
format_stamp: "Formatted at 2026-08-08 00:45:06 on dist-test-slave-kvfs"
server_key: "3f42c6d6e127af44c4182af7e7d8bc43"
server_key_iv: "ad6078a87f831cfbf82e48b3e7fce442"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:06.110751 7025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:06.152693 7025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:06.156379 7025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:06.156605 7025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:06.156895 7025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:06.157478 7025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:06.157537 7025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:06.157578 7025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:06.157634 7025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:06.167483 7025 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:36997
I20260808 00:45:06.168268 7025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:45:06.168802 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 7025
I20260808 00:45:06.168932 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:45:06.169205 20901 external_mini_cluster.cc:1428] Setting key 1568ecfccb0d856eee3200ddcdf29669
I20260808 00:45:06.169472 7209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:36997 every 8 connection(s)
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:06.183687 6841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:06.170616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:52465 (local address 127.20.105.126:39271)
0808 00:45:06.170679 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:06.170682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:06.171044 (+ 362us) server_negotiation.cc:407] Connection header received
0808 00:45:06.171941 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:06.171942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:06.171959 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:06.171995 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:06.173188 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:06.174075 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:06.175887 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:06.176242 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:06.176333 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:06.179602 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:06.179604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:06.179604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:06.179616 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:06.181703 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:06.182314 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:06.182315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:06.182316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:06.182387 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:06.182751 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:06.182752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:06.182753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:06.182923 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:06.182970 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:06.183444 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 00:45:06.183559 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:45:06.184782 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:06.170643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:39271 (local address 127.20.105.67:52465)
0808 00:45:06.170877 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:06.170941 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:06.171791 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:06.172286 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:06.172293 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:06.172660 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:06.173050 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:06.173059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:06.174220 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:06.174221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:06.175769 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:06.175774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:06.176374 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:06.176376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:06.176624 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:06.177414 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:06.177427 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:06.179449 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:06.181807 (+ 2358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:06.181820 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:06.181824 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:06.182213 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:06.182460 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:06.182461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:06.182461 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:06.182647 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:06.183018 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:06.183025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:06.183261 (+ 236us) client_negotiation.cc:769] Sending connection context
0808 00:45:06.184234 (+ 973us) client_negotiation.cc:240] Negotiation successful
0808 00:45:06.184468 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":111,"threads_started":1}
I20260808 00:45:06.185838 7210 heartbeater.cc:347] Connected to a master server at 127.20.105.126:39271
I20260808 00:45:06.185983 7210 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:06.186267 7210 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:06.186838 6555 ts_manager.cc:194] Registered new tserver with Master: 4fe81f3b80d342e3939db52e5592df1d (127.20.105.67:36997)
I20260808 00:45:06.187414 6555 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:52465
I20260808 00:45:06.193110 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:45:06.194666 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 6667
I20260808 00:45:06.219020 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 6845
I20260808 00:45:06.244942 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 7025
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149905, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:06.264456 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 6508
I20260808 00:45:06.296409 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:39271
--webserver_interface=127.20.105.126
--webserver_port=39479
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:39271
--ranger_config_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:55765/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.105.84:55765/kms with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:06.409082 7215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:06.409386 7215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:06.409415 7215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:06.410830 7215 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:06.410892 7215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:06.410908 7215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:06.410921 7215 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:06.410943 7215 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:06.413125 7215 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.105.84:55765/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:39271
--ranger_config_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:39271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=39479
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:06.413486 7215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:06.413792 7215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:06.419824 7224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:06.419844 7226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:06.419833 7223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:06.420205 7215 server_base.cc:1034] running on GCE node
I20260808 00:45:06.420588 7215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:06.421030 7215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:06.422200 7215 hybrid_clock.cc:648] HybridClock initialized: now 1786149906422179 us; error 41 us; skew 500 ppm
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:06.425474 7215 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:06.426260 7215 webserver.cc:460] Webserver started at http://127.20.105.126:39479/ using document root <none> and password file <none>
I20260808 00:45:06.426589 7215 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:06.426688 7215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:06.512168 7215 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:45:06.513317 7236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:06.513618 7215 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:06.513754 7215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "2707680cff2842068b20eeb7895e41f1"
format_stamp: "Formatted at 2026-08-08 00:45:04 on dist-test-slave-kvfs"
server_key: "f1b0b8ba24404eb4cd0629482f2cf172"
server_key_iv: "d82dfb1debbcd4d9e0f36d15884909f2"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:06.514365 7215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:06.523721 7215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:06.525995 7215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:06.526368 7215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:06.531992 7215 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:39271
I20260808 00:45:06.532058 7309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:39271 every 8 connection(s)
I20260808 00:45:06.532567 7215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:06.534154 7311 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:45:06.534988 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Bootstrap starting.
I20260808 00:45:06.538002 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 7215
I20260808 00:45:06.538411 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:55765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:34119
--local_ip_for_outbound_sockets=127.20.105.65
--tserver_master_addrs=127.20.105.126:39271
--webserver_port=45103
--webserver_interface=127.20.105.65
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.105.84:55765/kms with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:06.545540 7311 log.cc:824] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:06.547930 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:45:06.548324 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Bootstrap complete.
I20260808 00:45:06.551276 7311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } }
I20260808 00:45:06.551602 7311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2707680cff2842068b20eeb7895e41f1, State: Initialized, Role: FOLLOWER
I20260808 00:45:06.551862 7311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } }
I20260808 00:45:06.551981 7311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:45:06.552062 7311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:45:06.552119 7311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:45:06.553514 7311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } }
I20260808 00:45:06.553669 7311 leader_election.cc:302] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2707680cff2842068b20eeb7895e41f1; no voters:
I20260808 00:45:06.553925 7311 leader_election.cc:288] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:45:06.554015 7315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:45:06.554286 7311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:06.554270 7315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [term 2 LEADER]: Becoming Leader. State: Replica: 2707680cff2842068b20eeb7895e41f1, State: Running, Role: LEADER
I20260808 00:45:06.554590 7315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } }
I20260808 00:45:06.554947 7318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "2707680cff2842068b20eeb7895e41f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } } }
I20260808 00:45:06.555022 7318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:06.555615 7321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2707680cff2842068b20eeb7895e41f1. Latest consensus state: current_term: 2 leader_uuid: "2707680cff2842068b20eeb7895e41f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2707680cff2842068b20eeb7895e41f1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 39271 } } }
I20260808 00:45:06.555686 7321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:06.556142 7327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:06.556293 7311 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 00:45:06.556813 7327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:45:06.560184 7327 catalog_manager.cc:1259] Loaded cluster ID: 7775c2af25d046c5a06abb869f870c3f
I20260808 00:45:06.560402 7327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:45:06.562995 7327 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:45:06.563395 7327 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Loaded TSK: 0
I20260808 00:45:06.563807 7327 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:45:06.658775 7313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:06.659111 7313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:06.659163 7313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:06.661013 7313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:06.661098 7313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:06.661151 7313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:45:06.663013 7313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.105.84:55765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:34119
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=45103
--tserver_master_addrs=127.20.105.126:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:06.663340 7313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:06.663628 7313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:06.669782 7369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:06.669817 7372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:06.669780 7370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:06.670467 7313 server_base.cc:1034] running on GCE node
I20260808 00:45:06.670763 7313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:06.671064 7313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:06.678494 7313 hybrid_clock.cc:648] HybridClock initialized: now 1786149906678448 us; error 45 us; skew 500 ppm
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:06.682225 7313 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:45:06.682871 7313 webserver.cc:460] Webserver started at http://127.20.105.65:45103/ using document root <none> and password file <none>
I20260808 00:45:06.683182 7313 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:06.683286 7313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:06.715776 7313 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 00:45:06.720834 7381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:06.721170 7313 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260808 00:45:06.721295 7313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "cde0511636fc45b7b6a229060ac632b3"
format_stamp: "Formatted at 2026-08-08 00:45:05 on dist-test-slave-kvfs"
server_key: "a82628d54d7fcafdcef63227bbbf4887"
server_key_iv: "0ed498a7fc22e897caf200896ee46ced"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:06.721942 7313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:06.758786 7313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:06.761754 7313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:06.761958 7313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:06.762349 7313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:06.762877 7313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:06.762956 7313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:06.763021 7313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:06.763069 7313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:06.771634 7313 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:34119
I20260808 00:45:06.772387 7313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:45:06.778044 7543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:34119 every 8 connection(s)
I20260808 00:45:06.779510 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 7313
I20260808 00:45:06.779939 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:55765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:34397
--local_ip_for_outbound_sockets=127.20.105.66
--tserver_master_addrs=127.20.105.126:39271
--webserver_port=37045
--webserver_interface=127.20.105.66
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.105.84:55765/kms with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:06.789651 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:06.773924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:39271 (local address 127.20.105.65:46439)
0808 00:45:06.774728 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:06.774790 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:06.775668 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:06.776531 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:06.776536 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:06.776823 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:06.777174 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:06.777182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:06.778968 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:06.778970 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:06.779971 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:06.779974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:06.780804 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:06.780805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:06.780940 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:06.781750 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:06.781764 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:06.783973 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:06.787005 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:06.787015 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:06.787019 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:06.787381 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:06.788077 (+ 696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:06.788079 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:06.788079 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:06.788243 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:06.788809 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:06.788816 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:06.788990 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 00:45:06.789089 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 00:45:06.789304 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":113,"threads_started":1}
I20260808 00:45:06.791026 7547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:06.774432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:46439 (local address 127.20.105.126:39271)
0808 00:45:06.774708 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:06.774722 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:06.775873 (+ 1151us) server_negotiation.cc:407] Connection header received
0808 00:45:06.776122 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:06.776130 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:06.776201 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:06.776349 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:06.777325 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:06.778841 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:06.780162 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:06.780667 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:06.780823 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:06.784203 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:06.784210 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:06.784210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:06.784228 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:06.786894 (+ 2666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:06.787807 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:06.787814 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:06.787814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:06.787933 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:06.788329 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:06.788330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:06.788330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:06.788719 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:06.790443 (+ 1724us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:06.790534 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 00:45:06.790716 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":105,"threads_started":1}
I20260808 00:45:06.792214 7544 heartbeater.cc:347] Connected to a master server at 127.20.105.126:39271
I20260808 00:45:06.792346 7544 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:06.792640 7544 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
W20260808 00:45:06.793972 7544 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:39271 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:45:06.794064 7544 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:06.794152 7544 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:06.794756 7544 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:06.794854 7544 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
W20260808 00:45:06.795264 7544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:45:06.909133 7548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:06.909469 7548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:06.909510 7548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:06.910902 7548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:06.910969 7548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:06.911017 7548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:45:06.913305 7548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.105.84:55765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:34397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=37045
--tserver_master_addrs=127.20.105.126:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:06.913770 7548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:06.914157 7548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:06.920886 7560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:06.920867 7554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:06.921304 7558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:06.921368 7548 server_base.cc:1034] running on GCE node
I20260808 00:45:06.921780 7548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:06.922185 7548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:06.923377 7548 hybrid_clock.cc:648] HybridClock initialized: now 1786149906923346 us; error 48 us; skew 500 ppm
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:06.926666 7548 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:45:06.927431 7548 webserver.cc:460] Webserver started at http://127.20.105.66:37045/ using document root <none> and password file <none>
I20260808 00:45:06.927774 7548 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:06.927871 7548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:45:06 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:06.952188 7548 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 00:45:06.957979 7577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:06.964150 7548 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260808 00:45:06.964457 7548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "af95958407c64107ad5aedba19bc8036"
format_stamp: "Formatted at 2026-08-08 00:45:05 on dist-test-slave-kvfs"
server_key: "26a25f1722163c6e1663306e341f0dce"
server_key_iv: "029bd31078e52d4a8380497fa5a0deee"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:06.965353 7548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:07.005373 7548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:07.008667 7548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:07.008924 7548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:07.009338 7548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:07.009876 7548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:07.009984 7548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:07.010033 7548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:07.010064 7548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:07.020520 7548 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:34397
I20260808 00:45:07.020625 7734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:34397 every 8 connection(s)
I20260808 00:45:07.021358 7548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:45:07.028439 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 7548
I20260808 00:45:07.028847 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.105.84:55765/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:36997
--local_ip_for_outbound_sockets=127.20.105.67
--tserver_master_addrs=127.20.105.126:39271
--webserver_port=33815
--webserver_interface=127.20.105.67
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.105.84:55765/kms with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:45:07 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149906, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:07.034289 7547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:07.022502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:39049 (local address 127.20.105.126:39271)
0808 00:45:07.022592 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:07.022595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:07.022880 (+ 285us) server_negotiation.cc:407] Connection header received
0808 00:45:07.023715 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:07.023716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:07.023736 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:07.023785 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:07.024619 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.025191 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.026508 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.026749 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.026859 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:07.030128 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:07.030131 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:07.030132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:07.030144 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:07.032279 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:07.032951 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:07.032953 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:07.032953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:07.033027 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:07.033349 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:07.033350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:07.033351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:07.033519 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:07.033580 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:07.034022 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 00:45:07.034143 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:45:07.035043 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:07.022493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:39271 (local address 127.20.105.66:39049)
0808 00:45:07.022713 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:07.022765 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:07.023577 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:07.023890 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:07.023900 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:07.024207 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:07.024514 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:07.024520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.025298 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.025299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:07.026412 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:07.026416 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.026970 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.026971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:07.027149 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:07.027922 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:07.027932 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:07.029982 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:07.032392 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:07.032403 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:07.032407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:07.032852 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:07.033107 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:07.033108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:07.033108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:07.033276 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:07.033631 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:07.033638 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:07.033854 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 00:45:07.034477 (+ 623us) client_negotiation.cc:240] Negotiation successful
0808 00:45:07.034700 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":137,"threads_started":1}
I20260808 00:45:07.035840 7735 heartbeater.cc:347] Connected to a master server at 127.20.105.126:39271
I20260808 00:45:07.035962 7735 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:07.036243 7735 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
W20260808 00:45:07.037066 7735 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:39271 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:45:07.037151 7735 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:07.037240 7735 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:07.037658 7735 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:07.037745 7735 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
W20260808 00:45:07.038105 7735 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:45:07.153784 7739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:07.154117 7739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:07.154171 7739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:07.156177 7739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:07.156268 7739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:07.156327 7739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:45:07.158728 7739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.105.84:55765/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:36997
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=33815
--tserver_master_addrs=127.20.105.126:39271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:07.159299 7739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:07.159673 7739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:07.166602 7748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:07.166611 7746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:07.167126 7744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:07.167853 7739 server_base.cc:1034] running on GCE node
I20260808 00:45:07.168179 7739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:07.168577 7739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:07.169791 7739 hybrid_clock.cc:648] HybridClock initialized: now 1786149907169768 us; error 44 us; skew 500 ppm
Aug 08 00:45:07 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149907, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:07.173679 7739 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:45:07.174467 7739 webserver.cc:460] Webserver started at http://127.20.105.67:33815/ using document root <none> and password file <none>
I20260808 00:45:07.174896 7739 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:07.175067 7739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:45:07 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149907, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:07.203289 7739 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:07.208734 7761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:07.209046 7739 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s
I20260808 00:45:07.209209 7739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "4fe81f3b80d342e3939db52e5592df1d"
format_stamp: "Formatted at 2026-08-08 00:45:06 on dist-test-slave-kvfs"
server_key: "3f42c6d6e127af44c4182af7e7d8bc43"
server_key_iv: "ad6078a87f831cfbf82e48b3e7fce442"
server_key_version: "kuduclusterkey@0"
I20260808 00:45:07.209946 7739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:07.234943 7739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:07.237769 7739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:07.238018 7739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:07.238366 7739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:07.238875 7739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:07.238930 7739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:07.238970 7739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:07.239071 7739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:07.250267 7739 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:36997
I20260808 00:45:07.251015 7739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:45:07.251508 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 7739
I20260808 00:45:07.252116 7918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:36997 every 8 connection(s)
Aug 08 00:45:07 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149907, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:07.265461 7547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:07.252134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.20.105.126:39271)
0808 00:45:07.252205 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:07.252208 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:07.252673 (+ 465us) server_negotiation.cc:407] Connection header received
0808 00:45:07.252787 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:07.252788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:07.252803 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:07.252868 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:07.256154 (+ 3286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.257108 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.259810 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.260148 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.260237 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:07.262052 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:07.262054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:07.262054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:07.262062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:07.264004 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:07.264464 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:07.264465 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:07.264466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:07.264536 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:07.264771 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:07.264772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:07.264773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:07.264917 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:07.265163 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:07.265205 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 00:45:07.265305 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:45:07.269335 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:07.253162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:38009 (local address 127.20.105.126:39271)
0808 00:45:07.255083 (+ 1921us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:07.255086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:07.255096 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:45:07.255129 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:07.255130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:07.255145 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:07.255244 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:07.256302 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.257150 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.258882 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.259245 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.259321 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:07.264349 (+ 5028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:07.264351 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:07.264351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:07.264360 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:07.267073 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:07.267816 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:07.267817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:07.267818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:07.267898 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:07.268318 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:07.268320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:07.268320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:07.268488 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:07.268574 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:07.269076 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 00:45:07.269188 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":1893,"thread_start_us":69,"threads_started":1}
I20260808 00:45:07.269492 7923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:07.252782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:39271 (local address 127.20.105.67:38009)
0808 00:45:07.253988 (+ 1206us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:07.254054 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:07.254848 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:07.255392 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:07.255398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:07.255749 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:07.256165 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:07.256173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.257282 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.257283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:07.258761 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:07.258765 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:07.259439 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:07.259440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:07.259607 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:07.262139 (+ 2532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:07.262152 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:07.264221 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:07.267214 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:07.267225 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:07.267228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:07.267696 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:07.267993 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:07.267994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:07.267995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:07.268225 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:07.268586 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:07.268595 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:07.268842 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 00:45:07.268998 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 00:45:07.269218 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1145,"thread_start_us":126,"threads_started":1}
I20260808 00:45:07.270156 7919 heartbeater.cc:347] Connected to a master server at 127.20.105.126:39271
I20260808 00:45:07.270316 7919 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:07.270578 7919 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
W20260808 00:45:07.271371 7919 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:39271 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:45:07.271473 7919 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:07.271556 7919 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:07.271914 7919 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:07.272017 7919 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
W20260808 00:45:07.272394 7919 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:45:07.795513 7544 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:07.795735 7544 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:07.796434 7263 ts_manager.cc:194] Registered new tserver with Master: cde0511636fc45b7b6a229060ac632b3 (127.20.105.65:34119)
I20260808 00:45:07.797564 7263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:46439
Aug 08 00:45:07 dist-test-slave-kvfs krb5kdc[5855](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149907, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:08.038343 7735 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:08.038568 7735 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:08.038993 7263 ts_manager.cc:194] Registered new tserver with Master: af95958407c64107ad5aedba19bc8036 (127.20.105.66:34397)
I20260808 00:45:08.039664 7263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:39049
I20260808 00:45:08.272639 7919 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:08.272877 7919 heartbeater.cc:511] Master 127.20.105.126:39271 requested a full tablet report, sending...
I20260808 00:45:08.273447 7263 ts_manager.cc:194] Registered new tserver with Master: 4fe81f3b80d342e3939db52e5592df1d (127.20.105.67:36997)
I20260808 00:45:08.274065 7263 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:38009
I20260808 00:45:08.278048 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:45:08.288712 7948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.279393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47236 (local address 127.20.105.126:39271)
0808 00:45:08.279547 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:08.279550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:08.279719 (+ 169us) server_negotiation.cc:407] Connection header received
0808 00:45:08.279780 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:08.279781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:08.279795 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:08.279880 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:08.280748 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.281610 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.282705 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.283059 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.283132 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.284975 (+ 1843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:08.284977 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:08.284977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:08.284987 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:08.286937 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:08.287419 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:08.287420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:08.287421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:08.287500 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:08.287755 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:08.287756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:08.287756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:08.287932 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:08.288067 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:08.288489 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 00:45:08.288587 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260808 00:45:08.294513 7263 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:45:08.297642 7263 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:45:08.308622 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.301831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38788 (local address 127.20.105.65:34119)
0808 00:45:08.302410 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:08.302416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:08.302718 (+ 302us) server_negotiation.cc:407] Connection header received
0808 00:45:08.302764 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:08.302766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:08.302785 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:08.302856 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:08.303271 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.304785 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.306755 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.307636 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.307720 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.307794 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:08.308432 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 00:45:08.308495 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":64,"threads_started":1}
I20260808 00:45:08.309166 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.301592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:34119 (local address 127.0.0.1:38788)
0808 00:45:08.302501 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:08.302549 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:08.302633 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:08.303068 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:08.303072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:08.303075 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:08.303162 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.303165 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.304883 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.304885 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.306656 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.306659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.307833 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.307834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.307927 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.307944 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 00:45:08.308035 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 00:45:08.309072 (+ 1037us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"thread_start_us":131,"threads_started":1}
I20260808 00:45:08.310369 7427 tablet_service.cc:1467] Processing CreateTablet for tablet fe1f65bc7b7742a98e3b20fc003af960 (DEFAULT_TABLE table=test-table [id=e2b99826876c4d06affe5404304ba77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:08.310905 7427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe1f65bc7b7742a98e3b20fc003af960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:08.313988 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.302059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39990 (local address 127.20.105.66:34397)
0808 00:45:08.304577 (+ 2518us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:08.304583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:08.305970 (+ 1387us) server_negotiation.cc:407] Connection header received
0808 00:45:08.306021 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:08.306023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:08.306045 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:08.306132 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:08.306508 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.307601 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.311886 (+ 4285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.313247 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.313356 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.313429 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:08.313820 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 00:45:08.313883 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2475,"thread_start_us":56,"threads_started":1}
I20260808 00:45:08.314203 7959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.301594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:34397 (local address 127.0.0.1:39990)
0808 00:45:08.305801 (+ 4207us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:08.305814 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:08.305888 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:08.306269 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:08.306270 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:08.306273 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:08.306325 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.306327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.310560 (+ 4233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.310562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.311753 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.311757 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.313527 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.313528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.313600 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.313606 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:45:08.313662 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:45:08.313713 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4184,"thread_start_us":126,"threads_started":1}
I20260808 00:45:08.314286 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.302165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41152 (local address 127.20.105.67:36997)
0808 00:45:08.303023 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:08.303028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:08.309686 (+ 6658us) server_negotiation.cc:407] Connection header received
0808 00:45:08.309748 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:08.309751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:08.309772 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:08.309856 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:08.310106 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.311519 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.312883 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.313756 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.313836 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.313896 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:08.314127 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 00:45:08.314185 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":813,"thread_start_us":72,"threads_started":1}
I20260808 00:45:08.314965 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.301909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:36997 (local address 127.0.0.1:41152)
0808 00:45:08.309553 (+ 7644us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:08.309565 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:08.309609 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:08.309984 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:08.309985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:08.309987 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:08.310024 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.310026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.311618 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.311619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.312767 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.312771 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.313935 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.313936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.313996 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.313999 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:45:08.314047 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:45:08.314892 (+ 845us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7630,"thread_start_us":79,"threads_started":1}
I20260808 00:45:08.315279 7646 tablet_service.cc:1467] Processing CreateTablet for tablet fe1f65bc7b7742a98e3b20fc003af960 (DEFAULT_TABLE table=test-table [id=e2b99826876c4d06affe5404304ba77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:08.315837 7828 tablet_service.cc:1467] Processing CreateTablet for tablet fe1f65bc7b7742a98e3b20fc003af960 (DEFAULT_TABLE table=test-table [id=e2b99826876c4d06affe5404304ba77b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:08.316082 7646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe1f65bc7b7742a98e3b20fc003af960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:08.316282 7828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe1f65bc7b7742a98e3b20fc003af960. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:08.320957 7968 tablet_bootstrap.cc:492] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: Bootstrap starting.
I20260808 00:45:08.321899 7968 tablet_bootstrap.cc:654] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:08.322362 7968 log.cc:824] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:08.323293 7968 tablet_bootstrap.cc:492] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: No bootstrap required, opened a new log
I20260808 00:45:08.323365 7968 ts_tablet_manager.cc:1397] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:45:08.324710 7966 tablet_bootstrap.cc:492] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3: Bootstrap starting.
I20260808 00:45:08.324880 7967 tablet_bootstrap.cc:492] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: Bootstrap starting.
Aug 08 00:45:08 dist-test-slave-kvfs krb5kdc[5855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149907, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for HTTP/127.20.105.84@KRBTEST.COM
I20260808 00:45:08.325626 7966 tablet_bootstrap.cc:654] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:08.325829 7967 tablet_bootstrap.cc:654] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:08.326417 7967 log.cc:824] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:08.327220 7966 log.cc:824] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:08.327229 7968 raft_consensus.cc:348] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.327558 7968 raft_consensus.cc:374] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:08.327695 7968 raft_consensus.cc:729] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fe81f3b80d342e3939db52e5592df1d, State: Initialized, Role: FOLLOWER
I20260808 00:45:08.327857 7968 consensus_queue.cc:260] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.328315 7968 ts_tablet_manager.cc:1428] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 00:45:08.328428 7919 heartbeater.cc:503] Master 127.20.105.126:39271 was elected leader, sending a full tablet report...
I20260808 00:45:08.329521 7967 tablet_bootstrap.cc:492] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: No bootstrap required, opened a new log
I20260808 00:45:08.329664 7967 ts_tablet_manager.cc:1397] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 00:45:08.333503 7967 raft_consensus.cc:348] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.333829 7967 raft_consensus.cc:374] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:08.333940 7967 raft_consensus.cc:729] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af95958407c64107ad5aedba19bc8036, State: Initialized, Role: FOLLOWER
I20260808 00:45:08.334206 7967 consensus_queue.cc:260] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.334646 7967 ts_tablet_manager.cc:1428] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 00:45:08.334834 7735 heartbeater.cc:503] Master 127.20.105.126:39271 was elected leader, sending a full tablet report...
I20260808 00:45:08.335762 7966 tablet_bootstrap.cc:492] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3: No bootstrap required, opened a new log
I20260808 00:45:08.335879 7966 ts_tablet_manager.cc:1397] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260808 00:45:08.338579 7966 raft_consensus.cc:348] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.338794 7966 raft_consensus.cc:374] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:08.338836 7966 raft_consensus.cc:729] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cde0511636fc45b7b6a229060ac632b3, State: Initialized, Role: FOLLOWER
I20260808 00:45:08.339121 7966 consensus_queue.cc:260] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.339514 7966 ts_tablet_manager.cc:1428] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:45:08.339689 7544 heartbeater.cc:503] Master 127.20.105.126:39271 was elected leader, sending a full tablet report...
I20260808 00:45:08.459779 7980 raft_consensus.cc:482] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:08.460080 7980 raft_consensus.cc:504] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.460759 7980 leader_election.cc:288] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af95958407c64107ad5aedba19bc8036 (127.20.105.66:34397), 4fe81f3b80d342e3939db52e5592df1d (127.20.105.67:36997)
I20260808 00:45:08.465435 7964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.461297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:36701 (local address 127.20.105.66:34397)
0808 00:45:08.461352 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:08.461356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:08.461544 (+ 188us) server_negotiation.cc:407] Connection header received
0808 00:45:08.461665 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:08.461667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:08.461678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:08.461709 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:08.461974 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.462801 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.464357 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.464961 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.465013 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.465024 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:08.465286 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 00:45:08.465334 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:45:08.465704 7965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.461439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:47949 (local address 127.20.105.67:36997)
0808 00:45:08.461505 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:08.461509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:08.461653 (+ 144us) server_negotiation.cc:407] Connection header received
0808 00:45:08.461777 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:08.461778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:08.461791 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:08.461827 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:08.462135 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.462891 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.464657 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.465295 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.465349 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.465358 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:08.465576 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 00:45:08.465624 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:45:08.465828 7982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.460998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:34397 (local address 127.20.105.65:36701)
0808 00:45:08.461461 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:08.461475 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:08.461578 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:08.461788 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:08.461789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:08.461791 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:08.461897 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.461899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.462904 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.462904 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.464250 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.464254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.465134 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.465135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.465207 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.465211 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:45:08.465664 (+ 453us) client_negotiation.cc:240] Negotiation successful
0808 00:45:08.465715 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":155,"threads_started":1}
I20260808 00:45:08.466213 7659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe1f65bc7b7742a98e3b20fc003af960" candidate_uuid: "cde0511636fc45b7b6a229060ac632b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af95958407c64107ad5aedba19bc8036" is_pre_election: true
I20260808 00:45:08.466414 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.461135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:36997 (local address 127.20.105.65:47949)
0808 00:45:08.461571 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:08.461584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:08.461686 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:08.461911 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:08.461912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:08.461914 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:08.462025 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.462027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.463024 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.463025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.464536 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:08.464540 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.465419 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.465420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:08.465498 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.465501 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:45:08.466265 (+ 764us) client_negotiation.cc:240] Negotiation successful
0808 00:45:08.466315 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":130,"threads_started":1}
I20260808 00:45:08.466449 7659 raft_consensus.cc:2454] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cde0511636fc45b7b6a229060ac632b3 in term 0.
I20260808 00:45:08.466708 7383 leader_election.cc:302] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af95958407c64107ad5aedba19bc8036, cde0511636fc45b7b6a229060ac632b3; no voters:
I20260808 00:45:08.466859 7980 raft_consensus.cc:2781] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:45:08.466903 7980 raft_consensus.cc:482] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:45:08.466925 7980 raft_consensus.cc:3037] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:08.468091 7980 raft_consensus.cc:504] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.468250 7980 leader_election.cc:288] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [CANDIDATE]: Term 1 election: Requested vote from peers af95958407c64107ad5aedba19bc8036 (127.20.105.66:34397), 4fe81f3b80d342e3939db52e5592df1d (127.20.105.67:36997)
I20260808 00:45:08.468185 7858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe1f65bc7b7742a98e3b20fc003af960" candidate_uuid: "cde0511636fc45b7b6a229060ac632b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fe81f3b80d342e3939db52e5592df1d" is_pre_election: true
I20260808 00:45:08.468379 7858 raft_consensus.cc:2454] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cde0511636fc45b7b6a229060ac632b3 in term 0.
I20260808 00:45:08.468487 7833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe1f65bc7b7742a98e3b20fc003af960" candidate_uuid: "cde0511636fc45b7b6a229060ac632b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fe81f3b80d342e3939db52e5592df1d"
I20260808 00:45:08.468580 7833 raft_consensus.cc:3037] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:08.469092 7659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe1f65bc7b7742a98e3b20fc003af960" candidate_uuid: "cde0511636fc45b7b6a229060ac632b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af95958407c64107ad5aedba19bc8036"
I20260808 00:45:08.469193 7659 raft_consensus.cc:3037] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:08.469605 7833 raft_consensus.cc:2454] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cde0511636fc45b7b6a229060ac632b3 in term 1.
I20260808 00:45:08.469903 7384 leader_election.cc:302] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fe81f3b80d342e3939db52e5592df1d, cde0511636fc45b7b6a229060ac632b3; no voters:
I20260808 00:45:08.469986 7980 raft_consensus.cc:2781] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:08.470158 7980 raft_consensus.cc:686] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [term 1 LEADER]: Becoming Leader. State: Replica: cde0511636fc45b7b6a229060ac632b3, State: Running, Role: LEADER
I20260808 00:45:08.470276 7980 consensus_queue.cc:237] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } }
I20260808 00:45:08.470413 7659 raft_consensus.cc:2454] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cde0511636fc45b7b6a229060ac632b3 in term 1.
I20260808 00:45:08.471341 7263 catalog_manager.cc:5495] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to cde0511636fc45b7b6a229060ac632b3 (127.20.105.65). New cstate: current_term: 1 leader_uuid: "cde0511636fc45b7b6a229060ac632b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cde0511636fc45b7b6a229060ac632b3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 34119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 } health_report { overall_health: UNKNOWN } } }
I20260808 00:45:08.494378 7962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.490907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.20.105.65:34119)
0808 00:45:08.490992 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:08.490994 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:08.491005 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:45:08.491060 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:08.491061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:08.491093 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:08.491184 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:45:08.491364 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.492232 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.493350 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:08.493696 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:08.493785 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:08.493946 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:45:08.494122 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:45:08.494224 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:08.494245 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 00:45:08.494271 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:45:08.497246 7833 raft_consensus.cc:1261] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 1 FOLLOWER]: Refusing update from remote peer cde0511636fc45b7b6a229060ac632b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:08.497311 7659 raft_consensus.cc:1261] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 1 FOLLOWER]: Refusing update from remote peer cde0511636fc45b7b6a229060ac632b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:08.497700 7980 consensus_queue.cc:1035] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fe81f3b80d342e3939db52e5592df1d" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36997 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:08.497869 7983 consensus_queue.cc:1035] T fe1f65bc7b7742a98e3b20fc003af960 P cde0511636fc45b7b6a229060ac632b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af95958407c64107ad5aedba19bc8036" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 34397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:08.500811 7986 mvcc.cc:205] Tried to move back new op lower bound from 7316070025201049600 to 7316070025094930432. Current Snapshot: MvccSnapshot[applied={T|T < 7316070025201049600}]
I20260808 00:45:08.502007 7988 mvcc.cc:205] Tried to move back new op lower bound from 7316070025201049600 to 7316070025094930432. Current Snapshot: MvccSnapshot[applied={T|T < 7316070025201049600}]
I20260808 00:45:08.503984 7989 mvcc.cc:205] Tried to move back new op lower bound from 7316070025201049600 to 7316070025094930432. Current Snapshot: MvccSnapshot[applied={T|T < 7316070025201049600}]
I20260808 00:45:08.506959 7263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:08.507113 7263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:08.508103 7263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2707680cff2842068b20eeb7895e41f1: Sending DeleteTablet for 3 replicas of tablet fe1f65bc7b7742a98e3b20fc003af960
I20260808 00:45:08.508518 7646 tablet_service.cc:1514] Processing DeleteTablet for tablet fe1f65bc7b7742a98e3b20fc003af960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:08 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:39990
I20260808 00:45:08.508559 7828 tablet_service.cc:1514] Processing DeleteTablet for tablet fe1f65bc7b7742a98e3b20fc003af960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:08 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:41152
I20260808 00:45:08.509107 7427 tablet_service.cc:1514] Processing DeleteTablet for tablet fe1f65bc7b7742a98e3b20fc003af960 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:08 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:38788
I20260808 00:45:08.509473 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 7313
I20260808 00:45:08.519002 8000 tablet_replica.cc:330] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: stopping tablet replica
I20260808 00:45:08.519235 8000 raft_consensus.cc:2229] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:08.519433 8000 raft_consensus.cc:2258] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:08.521132 8000 ts_tablet_manager.cc:1905] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:08.523485 8000 ts_tablet_manager.cc:1918] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:08.523653 8000 log.cc:1196] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/fe1f65bc7b7742a98e3b20fc003af960
I20260808 00:45:08.524008 8000 ts_tablet_manager.cc:1939] T fe1f65bc7b7742a98e3b20fc003af960 P af95958407c64107ad5aedba19bc8036: Deleting consensus metadata
I20260808 00:45:08.524734 7238 catalog_manager.cc:4841] TS af95958407c64107ad5aedba19bc8036 (127.20.105.66:34397): tablet fe1f65bc7b7742a98e3b20fc003af960 (table test-table [id=e2b99826876c4d06affe5404304ba77b]) successfully deleted
I20260808 00:45:08.528857 8001 tablet_replica.cc:330] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: stopping tablet replica
I20260808 00:45:08.529137 8001 raft_consensus.cc:2229] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:08.529379 8001 raft_consensus.cc:2258] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:08.530040 8001 ts_tablet_manager.cc:1905] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:08.532289 8001 ts_tablet_manager.cc:1918] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:08.532439 8001 log.cc:1196] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/fe1f65bc7b7742a98e3b20fc003af960
I20260808 00:45:08.532899 8001 ts_tablet_manager.cc:1939] T fe1f65bc7b7742a98e3b20fc003af960 P 4fe81f3b80d342e3939db52e5592df1d: Deleting consensus metadata
I20260808 00:45:08.533541 7241 catalog_manager.cc:4841] TS 4fe81f3b80d342e3939db52e5592df1d (127.20.105.67:36997): tablet fe1f65bc7b7742a98e3b20fc003af960 (table test-table [id=e2b99826876c4d06affe5404304ba77b]) successfully deleted
W20260808 00:45:08.535573 7237 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:34119 (error 108)
I20260808 00:45:08.536094 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 7548
I20260808 00:45:08.536351 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:08.536181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:34119 (local address 127.0.0.1:38802)
0808 00:45:08.536235 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:08.536304 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:34119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 00:45:08.552315 7237 catalog_manager.cc:4584] TS cde0511636fc45b7b6a229060ac632b3 (127.20.105.65:34119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fe1f65bc7b7742a98e3b20fc003af960: Network error: Client connection negotiation failed: client connection to 127.20.105.65:34119: connect: Connection refused (error 111)
I20260808 00:45:08.559821 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 7739
I20260808 00:45:08.598301 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 7215
I20260808 00:45:08.612721 20901 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 00:45:08.960803 20901 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 00:45:08.960932 20901 mini_ranger.cc:66] Stopping Ranger...
I20260808 00:45:09.317000 20901 mini_ranger.cc:68] Stopped Ranger
2026-08-08 00:45:09.317 UTC [5906] LOG: received smart shutdown request
2026-08-08 00:45:09.322 UTC [5906] LOG: background worker "logical replication launcher" (PID 5918) exited with exit code 1
2026-08-08 00:45:09.322 UTC [5912] LOG: shutting down
2026-08-08 00:45:09.529 UTC [5906] LOG: database system is shut down
2026-08-08T00:45:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (57263 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:45:09 dist-test-slave-kvfs krb5kdc[8014](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:45:09 dist-test-slave-kvfs krb5kdc[8014](info): set up 2 sockets
Aug 08 00:45:09 dist-test-slave-kvfs krb5kdc[8014](info): commencing operation
krb5kdc: starting...
W20260808 00:45:10.937021 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:45:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:45:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:45:11.088218 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33861
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:33861,127.20.105.125:39943,127.20.105.124:37665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:11.211553 8033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:11.211894 8033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:11.211936 8033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:11.213240 8033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:11.213321 8033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:11.213346 8033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:11.213367 8033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:11.213398 8033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:11.215119 8033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:33861,127.20.105.125:39943,127.20.105.124:37665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:11.215473 8033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:11.215764 8033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:11.221544 8041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:11.221541 8044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:11.221511 8039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:11.221902 8033 server_base.cc:1034] running on GCE node
I20260808 00:45:11.222229 8033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:11.222602 8033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:11.223785 8033 hybrid_clock.cc:648] HybridClock initialized: now 1786149911223765 us; error 43 us; skew 500 ppm
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:11.226792 8033 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:11.227355 8033 webserver.cc:460] Webserver started at http://127.20.105.126:45769/ using document root <none> and password file <none>
I20260808 00:45:11.227664 8033 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:11.227717 8033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:11.227900 8033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:11.229264 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "d04aea24d9e14a6fbcf50af24f891ac5"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "519bf7d85a0da8387c41e5ee1894c8e1"
server_key_iv: "403fdf0d931dc732f6f2707ed7f1298e"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.229717 8033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "d04aea24d9e14a6fbcf50af24f891ac5"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "519bf7d85a0da8387c41e5ee1894c8e1"
server_key_iv: "403fdf0d931dc732f6f2707ed7f1298e"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.231320 8033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:45:11.232249 8052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:11.232479 8033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:45:11.232600 8033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "d04aea24d9e14a6fbcf50af24f891ac5"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "519bf7d85a0da8387c41e5ee1894c8e1"
server_key_iv: "403fdf0d931dc732f6f2707ed7f1298e"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.232683 8033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:11.250212 8033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:11.253100 8033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:11.253305 8033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:11.258663 8033 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:33861
I20260808 00:45:11.258713 8127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:33861 every 8 connection(s)
I20260808 00:45:11.259196 8033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:11.259434 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 8033
I20260808 00:45:11.259578 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:45:11.259807 20901 external_mini_cluster.cc:1428] Setting key 7bb1ddf270278212566bcfc432bee2cb
I20260808 00:45:11.260476 8130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:11.262465 8130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.105.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.125@KRBTEST.COM" created.
I20260808 00:45:11.273727 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.263428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47570 (local address 127.20.105.126:33861)
0808 00:45:11.263651 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.263662 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.263814 (+ 152us) server_negotiation.cc:407] Connection header received
0808 00:45:11.264549 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.264551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.264570 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.264688 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.265330 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.266312 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.267302 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.267677 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.267820 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.270230 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.270243 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.270244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.270251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.271706 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.272190 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.272193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.272193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.272289 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.272586 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.272587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.272587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.272840 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.272941 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.273239 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.273460 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":102,"server-negotiator.queue_time_us":182,"thread_start_us":118,"threads_started":1}
I20260808 00:45:11.274372 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.263418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33861 (local address 127.0.0.1:47570)
0808 00:45:11.263642 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.263690 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.264358 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.264695 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.264698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.265036 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.265223 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.265227 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.266419 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.266420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.267197 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.267199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.267785 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.267786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.267856 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.268570 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.268580 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.270092 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.271808 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.271816 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.271818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.272085 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.272376 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.272377 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.272377 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.272508 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.272939 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.272946 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.273038 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.273139 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.273354 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260808 00:45:11.275306 8130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:45:11.275775 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.275625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:57238)
0808 00:45:11.275680 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.275738 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:45:11.275908 8057 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
I20260808 00:45:11.276365 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.276287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:57240)
0808 00:45:11.276315 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.276342 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260808 00:45:11.276501 8130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:39943: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab.
Entry for principal kudu/127.20.105.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab.
I20260808 00:45:11.291965 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.125:39943
--webserver_interface=127.20.105.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab
--principal=kudu/127.20.105.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:33861,127.20.105.125:39943,127.20.105.124:37665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:11.325809 8130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } attempt: 1
I20260808 00:45:11.326776 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.326590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:49870)
0808 00:45:11.326675 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.326730 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 00:45:11.327235 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.327115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:49882)
0808 00:45:11.327168 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.327207 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:45:11.327345 8130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:39943: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
W20260808 00:45:11.389266 8143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:11.389557 8143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:11.389599 8143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:11.390806 8143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:11.390882 8143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:11.390908 8143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:11.390929 8143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:11.390961 8143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:11.392650 8143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:33861,127.20.105.125:39943,127.20.105.124:37665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab
--principal=kudu/127.20.105.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.125:39943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.105.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:11.392977 8143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:11.393270 8143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:45:11.395545 8130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } attempt: 2
I20260808 00:45:11.396286 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.396113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:49888)
0808 00:45:11.396188 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.396240 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 00:45:11.396729 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.396608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:49902)
0808 00:45:11.396665 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.396703 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:45:11.396838 8130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:39943: Network error: Client connection negotiation failed: client connection to 127.20.105.125:39943: connect: Connection refused (error 111)
W20260808 00:45:11.399621 8151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:11.399771 8143 server_base.cc:1034] running on GCE node
W20260808 00:45:11.399590 8153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:11.399621 8150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:11.400344 8143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:11.400694 8143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:11.401889 8143 hybrid_clock.cc:648] HybridClock initialized: now 1786149911401873 us; error 34 us; skew 500 ppm
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:11.405187 8143 init.cc:377] Logged in from keytab as kudu/127.20.105.125@KRBTEST.COM (short username kudu)
I20260808 00:45:11.405753 8143 webserver.cc:460] Webserver started at http://127.20.105.125:38095/ using document root <none> and password file <none>
I20260808 00:45:11.406075 8143 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:11.406128 8143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:11.406289 8143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:11.407621 8143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/data/instance:
uuid: "bbb2e247fd10406a91f6e26003cdd5d1"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "075e43f9fb9223b2b61fb9d6a9804c5c"
server_key_iv: "e567cc19830277dc9b14f0516e5fd756"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.408103 8143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/wal/instance:
uuid: "bbb2e247fd10406a91f6e26003cdd5d1"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "075e43f9fb9223b2b61fb9d6a9804c5c"
server_key_iv: "e567cc19830277dc9b14f0516e5fd756"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.409823 8143 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:11.410652 8162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:11.410863 8143 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:11.410975 8143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/wal
uuid: "bbb2e247fd10406a91f6e26003cdd5d1"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "075e43f9fb9223b2b61fb9d6a9804c5c"
server_key_iv: "e567cc19830277dc9b14f0516e5fd756"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.411057 8143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:11.432797 8143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:11.435438 8143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:11.435628 8143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:11.441520 8143 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.125:39943
I20260808 00:45:11.441601 8240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.125:39943 every 8 connection(s)
I20260808 00:45:11.442126 8143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
I20260808 00:45:11.443444 8243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:11.445551 8243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:45:11.450106 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 8143
I20260808 00:45:11.450279 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-1/wal/instance
I20260808 00:45:11.450506 20901 external_mini_cluster.cc:1428] Setting key 2d7469d3d1b809989c3593fc83aa6676
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:11.457859 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.446546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.20.105.126:33861)
0808 00:45:11.446632 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.446635 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.446790 (+ 155us) server_negotiation.cc:407] Connection header received
0808 00:45:11.447360 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.447362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.447381 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.447461 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.448407 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.448991 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.450208 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.450595 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.450689 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.454085 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.454088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.454089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.454098 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.455850 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.456577 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.456579 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.456580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.456664 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.456998 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.457000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.457000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.457164 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.457253 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.457613 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.457731 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:45:11.458112 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.446432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33861 (local address 127.0.0.1:39572)
0808 00:45:11.446638 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.446690 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.447238 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.447604 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.447608 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.447945 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.448294 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.448299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.449120 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.449121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.450088 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.450091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.450695 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.450697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.450832 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.451754 (+ 922us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.451770 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.453936 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.456096 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.456110 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.456119 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.456435 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.456761 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.456762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.456763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.456903 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.457241 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.457250 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.457444 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.457574 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.457794 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260808 00:45:11.458918 8243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.105.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.124@KRBTEST.COM" created.
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:45:11.469381 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.459301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:49908)
0808 00:45:11.459547 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.459559 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.459622 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.459800 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.459801 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.459989 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.460126 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.460129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.461341 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.461342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.462187 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.462190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.462805 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.462807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.462873 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.463402 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.463407 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.465853 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.468123 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.468124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.468125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.468300 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.468575 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.468576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.468577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.468644 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.469065 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.469066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.469127 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.469191 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.469289 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226}
I20260808 00:45:11.469568 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.459381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.20.105.125:39943)
0808 00:45:11.459568 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.459573 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.459615 (+ 42us) server_negotiation.cc:407] Connection header received
0808 00:45:11.459685 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.459687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.459701 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.459780 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.460303 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.461191 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.462291 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.462659 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.462765 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.465956 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.465965 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.465966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.465973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.467998 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.468390 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.468394 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.468395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.468475 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.468716 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.468717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.468717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.468961 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.469054 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.469261 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.469363 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260808 00:45:11.470378 8243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:45:11.470850 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.470700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43636)
0808 00:45:11.470756 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.470815 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:45:11.471019 8166 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
I20260808 00:45:11.471598 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.471462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43648)
0808 00:45:11.471529 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.471565 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 00:45:11.471787 8243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37665: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab.
Entry for principal kudu/127.20.105.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab.
I20260808 00:45:11.485338 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.124:37665
--webserver_interface=127.20.105.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab
--principal=kudu/127.20.105.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:33861,127.20.105.125:39943,127.20.105.124:37665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:11.487993 8130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } attempt: 3
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:45:11.496248 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.488578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.20.105.125:39943)
0808 00:45:11.488649 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.488652 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.488665 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:45:11.488735 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.488736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.488750 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.488822 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.489218 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.489781 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.490572 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.490797 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.490884 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.493487 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.493489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.493491 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.493504 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.495020 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.495381 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.495382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.495383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.495446 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.495654 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.495655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.495656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.495783 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.495865 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.496039 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.496131 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:45:11.496248 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.488497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:49916)
0808 00:45:11.488578 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.488589 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.488656 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.488847 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.488848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.489037 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.489125 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.489127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.489871 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.489872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.490470 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.490473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.490876 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.490878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.490971 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.491627 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.491633 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.493343 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.495125 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.495127 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.495127 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.495306 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.495525 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.495525 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.495526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.495590 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.495857 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.495858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.495913 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.495996 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.496130 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 00:45:11.496729 8130 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:45:11.497179 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.497050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43654)
0808 00:45:11.497106 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.497139 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 00:45:11.497534 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.497434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43668)
0808 00:45:11.497483 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.497508 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:45:11.497673 8130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37665: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
I20260808 00:45:11.521088 8243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } attempt: 1
I20260808 00:45:11.521812 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.521635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43682)
0808 00:45:11.521711 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.521767 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 00:45:11.522236 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.522136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43692)
0808 00:45:11.522186 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.522215 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:45:11.522344 8243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37665: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
I20260808 00:45:11.528852 8130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } attempt: 1
I20260808 00:45:11.529513 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.529344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43702)
0808 00:45:11.529425 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.529470 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 00:45:11.530049 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.529936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43704)
0808 00:45:11.529991 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.530021 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 00:45:11.530189 8130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37665: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
W20260808 00:45:11.585353 8256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:11.585610 8256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:11.585634 8256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:11.586839 8256 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:11.586885 8256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:11.586900 8256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:11.586941 8256 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:11.586968 8256 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:11.588647 8256 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:33861,127.20.105.125:39943,127.20.105.124:37665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab
--principal=kudu/127.20.105.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.124:37665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.105.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:11.588958 8256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:11.589234 8256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:45:11.590508 8243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } attempt: 2
I20260808 00:45:11.591147 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.590961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43716)
0808 00:45:11.591045 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.591102 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 00:45:11.591567 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.591461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43720)
0808 00:45:11.591512 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.591544 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:45:11.591681 8243 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37665: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
W20260808 00:45:11.595150 8264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:11.595157 8268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:11.595157 8270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:11.595326 8256 server_base.cc:1034] running on GCE node
I20260808 00:45:11.595644 8256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:11.595974 8256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:11.597206 8256 hybrid_clock.cc:648] HybridClock initialized: now 1786149911597197 us; error 33 us; skew 500 ppm
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:11.600096 8256 init.cc:377] Logged in from keytab as kudu/127.20.105.124@KRBTEST.COM (short username kudu)
I20260808 00:45:11.600613 8256 webserver.cc:460] Webserver started at http://127.20.105.124:37919/ using document root <none> and password file <none>
I20260808 00:45:11.600870 8256 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:11.600942 8256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:11.601107 8256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:11.602411 8256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/data/instance:
uuid: "0dd12f57f3ad40359b21047871667577"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "51df3a67439b04314beb69e644515c0d"
server_key_iv: "d5a56067aa89a5954d0228c8e7aec3e2"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.602826 8256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/wal/instance:
uuid: "0dd12f57f3ad40359b21047871667577"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "51df3a67439b04314beb69e644515c0d"
server_key_iv: "d5a56067aa89a5954d0228c8e7aec3e2"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.604426 8256 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:45:11.605284 8282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:11.605356 8130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } attempt: 2
I20260808 00:45:11.605530 8256 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:11.605690 8256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/wal
uuid: "0dd12f57f3ad40359b21047871667577"
format_stamp: "Formatted at 2026-08-08 00:45:11 on dist-test-slave-kvfs"
server_key: "51df3a67439b04314beb69e644515c0d"
server_key_iv: "d5a56067aa89a5954d0228c8e7aec3e2"
server_key_version: "encryptionkey@0"
I20260808 00:45:11.605790 8256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:11.605948 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.605770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43736)
0808 00:45:11.605849 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.605897 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 00:45:11.606383 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.606274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43750)
0808 00:45:11.606324 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.606359 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 00:45:11.606501 8130 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37665: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37665: connect: Connection refused (error 111)
I20260808 00:45:11.626478 8256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:11.629652 8256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:11.629842 8256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:11.634886 8256 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.124:37665
I20260808 00:45:11.634958 8362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.124:37665 every 8 connection(s)
I20260808 00:45:11.635423 8256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
I20260808 00:45:11.636643 8363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:11.638576 8363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:45:11.643083 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 8256
I20260808 00:45:11.643246 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149729697709-20901-0/minicluster-data/master-2/wal/instance
I20260808 00:45:11.643469 20901 external_mini_cluster.cc:1428] Setting key 7bf5104d69b12e1b61c143cc6e7b7627
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:45:11.650740 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.639397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33861 (local address 127.0.0.1:39586)
0808 00:45:11.639625 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.639688 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.640316 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.640671 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.640675 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.641028 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.641342 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.641347 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.642116 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.642117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.643076 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.643080 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.643576 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.643577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.643733 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.644452 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.644463 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.646066 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.648165 (+ 2099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.648174 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.648177 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.648539 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.649412 (+ 873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.649413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.649413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.649560 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.649896 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.649905 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.650092 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.650215 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.650411 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260808 00:45:11.651170 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.639647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39586 (local address 127.20.105.126:33861)
0808 00:45:11.639725 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.639727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.639804 (+ 77us) server_negotiation.cc:407] Connection header received
0808 00:45:11.640467 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.640469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.640486 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.640576 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.641457 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.642013 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.643176 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.643471 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.643565 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.646176 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.646178 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.646179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.646188 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.648013 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.649234 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.649236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.649236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.649313 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.649640 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.649641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.649641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.649800 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.649867 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.650947 (+ 1080us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.651047 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:45:11.651852 8363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:45:11.653522 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.643956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43756 (local address 127.20.105.124:37665)
0808 00:45:11.644218 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.644223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.644240 (+ 17us) server_negotiation.cc:407] Connection header received
0808 00:45:11.644266 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.644268 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.644309 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.644409 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.645086 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.645916 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.646898 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.647245 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.647330 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.650207 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.650215 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.650215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.650222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.652247 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.652653 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.652656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.652657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.652729 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.652950 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.652951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.652951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.653131 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.653287 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.653333 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.653415 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":157,"threads_started":1}
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:11.659439 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.652340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49922 (local address 127.20.105.125:39943)
0808 00:45:11.652402 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.652405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.652417 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:45:11.652449 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.652450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.652462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.652518 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.652885 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.653641 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.654306 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.654576 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.654634 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.656616 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.656617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.656618 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.656626 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.658311 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.658654 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.658655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.658655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.658721 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.658944 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.658945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.658946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.659060 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.659115 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.659285 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.659349 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:45:11.659469 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.652190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:39943 (local address 127.0.0.1:49922)
0808 00:45:11.652238 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.652251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.652392 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.652592 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.652594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.652739 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.652797 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.652798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.653718 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.653719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.654228 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.654230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.654646 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.654647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.654712 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.655122 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.655127 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.656527 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.658416 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.658417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.658417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.658579 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.658793 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.658793 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.658794 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.658873 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.659129 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.659131 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.659180 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.659268 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.659360 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 00:45:11.660045 8363 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:45:11.662846 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.654794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39590 (local address 127.20.105.126:33861)
0808 00:45:11.654854 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.654858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.654868 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:45:11.654891 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.654892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.654902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.654954 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.655586 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.656108 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.657137 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.657325 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.657410 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.659919 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.659921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.659921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.659929 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.661526 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.661903 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.661904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.661904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.661969 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.662187 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.662188 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.662189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.662311 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.662560 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.662606 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.662774 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:45:11.667040 8366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.660361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43758)
0808 00:45:11.660414 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.660423 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.660462 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.660651 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.660651 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.660789 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.660816 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.660817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.661465 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.661466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.661930 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.661931 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.662303 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.662304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.662354 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.662773 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.662778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.664170 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.665857 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.665858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.665858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.666096 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.666290 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.666290 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.666291 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.666364 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.666693 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.666694 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.666744 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.666819 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.666930 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 00:45:11.667127 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.660488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.20.105.124:37665)
0808 00:45:11.660542 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.660544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.660551 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:45:11.660574 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.660575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.660584 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.660649 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.660894 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.661373 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.662023 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.662233 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.662306 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.664264 (+ 1958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.664266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.664266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.664273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.665768 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.666171 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.666172 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.666172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.666225 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.666446 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.666446 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.666447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.666591 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.666661 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.666859 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.666997 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":38}
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:45:11.669018 8363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577: Bootstrap starting.
I20260808 00:45:11.670073 8363 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:11.670413 8363 log.cc:824] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:11.671293 8363 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577: No bootstrap required, opened a new log
I20260808 00:45:11.671944 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.663569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49938 (local address 127.20.105.125:39943)
0808 00:45:11.663614 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.663616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.663625 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:45:11.663647 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.663648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.663658 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.663714 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.664338 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.665003 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.666018 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.666342 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.666417 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.669079 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.669080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.669080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.669088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.670719 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.671062 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.671063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.671063 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.671124 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.671335 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.671336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.671336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.671474 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.671702 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.671743 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.671819 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:45:11.673146 8363 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:11.673322 8363 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:11.673390 8363 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dd12f57f3ad40359b21047871667577, State: Initialized, Role: FOLLOWER
I20260808 00:45:11.673707 8363 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:11.673990 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:45:11.674069 8371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:11.674147 8363 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:11.674158 8371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:45:11.676271 8393 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:45:11.676815 8393 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:45:11.677186 8256 master_runner.cc:186] Error getting master registration for 127.20.105.126:33861: OK, instance_id { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" instance_seqno: 1786149911257144 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:45:11.680440 8256 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:45:11.682825 8243 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } attempt: 3
I20260808 00:45:11.685364 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.676367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.20.105.126:33861)
0808 00:45:11.676503 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.676506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.676514 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:45:11.676551 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.676552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.676564 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.676616 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.677659 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.678447 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.679447 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.679742 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.679805 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.681477 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.681479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.681480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.681488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.683284 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.683691 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.683694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.683694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.683763 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.683971 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.683972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.683972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.684156 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.684920 (+ 764us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.685138 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.685223 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260808 00:45:11.685897 8250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.675879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.20.105.125:39943)
0808 00:45:11.675924 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.675926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.676158 (+ 232us) server_negotiation.cc:407] Connection header received
0808 00:45:11.676221 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.676222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.676234 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.676283 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.677666 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.678370 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.679443 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.679724 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.679769 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.681658 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.681659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.681660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.681667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.684211 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.684693 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.684695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.684695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.684756 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.684982 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.684983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.684984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.685387 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.685469 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.685692 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.685779 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:45:11.688172 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.676417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.20.105.124:37665)
0808 00:45:11.676670 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.676673 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.676680 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:45:11.676992 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.676993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.677003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.677063 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.681164 (+ 4101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.681865 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.682846 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.683141 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.683210 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.684816 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.684818 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.684818 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.684827 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.686873 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.687238 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.687239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.687240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.687306 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.687513 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.687514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.687515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.687688 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.687796 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.687943 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.688066 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231}
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:45:11.695308 8245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.683144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43786)
0808 00:45:11.683312 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.683325 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.683385 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.688588 (+ 5203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.688590 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.688768 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.688825 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.688826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.689458 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.689459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.689974 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.689975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.690368 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.690369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.690429 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.690842 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.690846 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.692608 (+ 1762us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.694338 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.694339 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.694339 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.694511 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.694716 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.694716 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.694717 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.694771 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.695054 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.695055 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.695093 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.695144 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.695231 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 00:45:11.695363 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.683484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.20.105.124:37665)
0808 00:45:11.688457 (+ 4973us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.688459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.688465 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:45:11.688479 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.688479 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.688487 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.688578 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.688901 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.689374 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.690077 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.690280 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.690367 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.692707 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.692709 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.692709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.692716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.694252 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.694586 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.694587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.694587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.694649 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.694832 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.694832 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.694833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.694989 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.695058 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.695187 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.695266 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4959,"thread_start_us":119,"threads_started":1}
I20260808 00:45:11.697116 8243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: Bootstrap starting.
I20260808 00:45:11.697960 8243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:11.698391 8243 log.cc:824] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:11.699365 8243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: No bootstrap required, opened a new log
I20260808 00:45:11.701228 8243 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:11.701408 8243 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:11.701462 8243 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb2e247fd10406a91f6e26003cdd5d1, State: Initialized, Role: FOLLOWER
I20260808 00:45:11.701632 8243 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:11.702025 8243 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:11.701989 8405 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:11.702109 8405 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:45:11.704198 8423 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:45:11.704319 8423 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:45:11.705687 8130 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } attempt: 3
I20260808 00:45:11.704483 8143 master_runner.cc:186] Error getting master registration for 127.20.105.126:33861: OK, instance_id { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" instance_seqno: 1786149911257144 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:45:11.707748 8143 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 00:45:11 dist-test-slave-kvfs krb5kdc[8014](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149911, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:45:11.716193 8135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.706061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37665 (local address 127.0.0.1:43802)
0808 00:45:11.706201 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:11.706216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:11.706263 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:11.707737 (+ 1474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:11.707739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:11.707975 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:11.708094 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.708097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.708879 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.708880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.709421 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:11.709423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.709880 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.709880 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:11.709946 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.710542 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:11.710548 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:11.711958 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:11.714966 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.714967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.714967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.715168 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.715402 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:11.715403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:11.715403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:11.715477 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:11.715775 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:11.715777 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:11.715834 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 00:45:11.715914 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 00:45:11.716070 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260808 00:45:11.716226 8402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:11.706346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.20.105.124:37665)
0808 00:45:11.707527 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:11.707529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:11.707539 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:45:11.707567 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:11.707568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:11.707600 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:11.708179 (+ 579us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:11.708231 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.708803 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.709532 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:11.709773 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:11.709869 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:11.712089 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:11.712091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:11.712091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:11.712099 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:11.714822 (+ 2723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.715246 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.715247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.715248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.715320 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:11.715542 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:11.715543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:11.715544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:11.715682 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:11.715759 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:11.715965 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 00:45:11.716083 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1112}
I20260808 00:45:11.718295 8130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: Bootstrap starting.
I20260808 00:45:11.719117 8130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:11.719480 8130 log.cc:824] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:11.720422 8130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: No bootstrap required, opened a new log
I20260808 00:45:11.722350 8130 raft_consensus.cc:348] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:11.722606 8130 raft_consensus.cc:374] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:11.722702 8130 raft_consensus.cc:729] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d04aea24d9e14a6fbcf50af24f891ac5, State: Initialized, Role: FOLLOWER
I20260808 00:45:11.722993 8130 consensus_queue.cc:260] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:11.723452 8429 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:11.723538 8130 sys_catalog.cc:564] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:11.723613 8429 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:45:11.725849 8447 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:45:11.726104 8447 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:45:11.728726 8033 master_runner.cc:263] No leader master found from master d04aea24d9e14a6fbcf50af24f891ac5
I20260808 00:45:11.728778 8033 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 00:45:11.754474 8060 tablet.cc:2369] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:45:11.880589 8288 tablet.cc:2369] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:45:11.936892 8170 tablet.cc:2369] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:45:12.139570 8371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:12.139756 8371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:12.140139 8371 leader_election.cc:288] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d04aea24d9e14a6fbcf50af24f891ac5 (127.20.105.126:33861), bbb2e247fd10406a91f6e26003cdd5d1 (127.20.105.125:39943)
I20260808 00:45:12.140555 8201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dd12f57f3ad40359b21047871667577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" is_pre_election: true
I20260808 00:45:12.140597 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dd12f57f3ad40359b21047871667577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" is_pre_election: true
I20260808 00:45:12.140769 8201 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dd12f57f3ad40359b21047871667577 in term 0.
I20260808 00:45:12.140800 8092 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0dd12f57f3ad40359b21047871667577 in term 0.
I20260808 00:45:12.141014 8286 leader_election.cc:302] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dd12f57f3ad40359b21047871667577, bbb2e247fd10406a91f6e26003cdd5d1; no voters:
I20260808 00:45:12.141207 8371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:45:12.141279 8371 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:45:12.141297 8371 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:12.142283 8371 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:12.142453 8371 leader_election.cc:288] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [CANDIDATE]: Term 1 election: Requested vote from peers d04aea24d9e14a6fbcf50af24f891ac5 (127.20.105.126:33861), bbb2e247fd10406a91f6e26003cdd5d1 (127.20.105.125:39943)
I20260808 00:45:12.142665 8092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dd12f57f3ad40359b21047871667577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d04aea24d9e14a6fbcf50af24f891ac5"
I20260808 00:45:12.142678 8201 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dd12f57f3ad40359b21047871667577" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbb2e247fd10406a91f6e26003cdd5d1"
I20260808 00:45:12.142777 8092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:12.142810 8201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:12.143707 8201 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dd12f57f3ad40359b21047871667577 in term 1.
I20260808 00:45:12.143723 8092 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0dd12f57f3ad40359b21047871667577 in term 1.
I20260808 00:45:12.143895 8286 leader_election.cc:302] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dd12f57f3ad40359b21047871667577, bbb2e247fd10406a91f6e26003cdd5d1; no voters:
I20260808 00:45:12.144027 8371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:12.144236 8371 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [term 1 LEADER]: Becoming Leader. State: Replica: 0dd12f57f3ad40359b21047871667577, State: Running, Role: LEADER
I20260808 00:45:12.144361 8371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } }
I20260808 00:45:12.144829 8449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd12f57f3ad40359b21047871667577. Latest consensus state: current_term: 1 leader_uuid: "0dd12f57f3ad40359b21047871667577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:12.144924 8449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:12.145196 8451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:12.145490 8451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:45:12.147125 8201 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [term 1 FOLLOWER]: Refusing update from remote peer 0dd12f57f3ad40359b21047871667577: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:12.147128 8092 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [term 1 FOLLOWER]: Refusing update from remote peer 0dd12f57f3ad40359b21047871667577: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:12.147473 8449 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:12.147631 8371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:12.148809 8429 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd12f57f3ad40359b21047871667577. Latest consensus state: current_term: 1 leader_uuid: "0dd12f57f3ad40359b21047871667577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:12.148926 8429 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:45:12.149773 8449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0dd12f57f3ad40359b21047871667577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:12.149863 8449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:12.150038 8429 sys_catalog.cc:455] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0dd12f57f3ad40359b21047871667577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:12.150133 8429 sys_catalog.cc:458] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:45:12.150205 8454 mvcc.cc:205] Tried to move back new op lower bound from 7316070040152084480 to 7316070040144064512. Current Snapshot: MvccSnapshot[applied={T|T < 7316070040152084480}]
I20260808 00:45:12.150297 8405 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dd12f57f3ad40359b21047871667577. Latest consensus state: current_term: 1 leader_uuid: "0dd12f57f3ad40359b21047871667577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:12.150403 8405 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:45:12.150461 8452 mvcc.cc:205] Tried to move back new op lower bound from 7316070040152084480 to 7316070040144064512. Current Snapshot: MvccSnapshot[applied={T|T < 7316070040152084480}]
I20260808 00:45:12.150656 8451 catalog_manager.cc:1347] Generated new cluster ID: 4aafa69f9f134edaba423648c3fc330e
I20260808 00:45:12.150709 8451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:45:12.151245 8449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0dd12f57f3ad40359b21047871667577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:12.151338 8449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:12.151592 8405 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0dd12f57f3ad40359b21047871667577" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d04aea24d9e14a6fbcf50af24f891ac5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33861 } } peers { permanent_uuid: "bbb2e247fd10406a91f6e26003cdd5d1" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 39943 } } peers { permanent_uuid: "0dd12f57f3ad40359b21047871667577" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37665 } } }
I20260808 00:45:12.151671 8405 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:45:12.186630 8451 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:45:12.187404 8451 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:45:12.192194 8451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dd12f57f3ad40359b21047871667577: Generated new TSK 0
I20260808 00:45:12.192521 8451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:45:12.705031 8423 catalog_manager.cc:1259] Loaded cluster ID: 4aafa69f9f134edaba423648c3fc330e
I20260808 00:45:12.705147 8423 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: loading cluster ID for follower catalog manager: success
I20260808 00:45:12.706075 8423 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: acquiring CA information for follower catalog manager: success
I20260808 00:45:12.706401 8423 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bbb2e247fd10406a91f6e26003cdd5d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:45:12.726827 8447 catalog_manager.cc:1259] Loaded cluster ID: 4aafa69f9f134edaba423648c3fc330e
I20260808 00:45:12.726940 8447 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: loading cluster ID for follower catalog manager: success
I20260808 00:45:12.727900 8447 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: acquiring CA information for follower catalog manager: success
I20260808 00:45:12.728343 8447 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d04aea24d9e14a6fbcf50af24f891ac5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:45:12.735025 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 8033
I20260808 00:45:12.753672 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 8143
I20260808 00:45:12.769471 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 8256
2026-08-08T00:45:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3126 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:45:12 dist-test-slave-kvfs krb5kdc[8472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:45:12 dist-test-slave-kvfs krb5kdc[8472](info): set up 2 sockets
Aug 08 00:45:12 dist-test-slave-kvfs krb5kdc[8472](info): commencing operation
krb5kdc: starting...
W20260808 00:45:14.088660 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.287s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:45:14 dist-test-slave-kvfs krb5kdc[8472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:45:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:45:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:45:14.243676 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:41745
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:43303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:41745
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:14.341022 8489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:14.341363 8489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:14.341406 8489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:14.342693 8489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:14.342773 8489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:14.342795 8489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:14.342818 8489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:14.342875 8489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:14.344610 8489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:43303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:41745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:41745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:14.344954 8489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:14.345239 8489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:14.351018 8495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:14.351074 8503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:14.350962 8496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:14.351547 8489 server_base.cc:1034] running on GCE node
I20260808 00:45:14.351874 8489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:14.352303 8489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:14.353497 8489 hybrid_clock.cc:648] HybridClock initialized: now 1786149914353468 us; error 51 us; skew 500 ppm
Aug 08 00:45:14 dist-test-slave-kvfs krb5kdc[8472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149914, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:14.356645 8489 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:14.357255 8489 webserver.cc:460] Webserver started at http://127.20.105.126:39165/ using document root <none> and password file <none>
I20260808 00:45:14.357551 8489 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:14.357630 8489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:14.357805 8489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:14.359153 8489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "30f615e293c844188e833853a53a4c95"
format_stamp: "Formatted at 2026-08-08 00:45:14 on dist-test-slave-kvfs"
server_key: "02e06c6fed8eb181c3fb422ff212900a"
server_key_iv: "960603964296e2d568d3ee2d916fcc4f"
server_key_version: "encryptionkey@0"
I20260808 00:45:14.359606 8489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "30f615e293c844188e833853a53a4c95"
format_stamp: "Formatted at 2026-08-08 00:45:14 on dist-test-slave-kvfs"
server_key: "02e06c6fed8eb181c3fb422ff212900a"
server_key_iv: "960603964296e2d568d3ee2d916fcc4f"
server_key_version: "encryptionkey@0"
I20260808 00:45:14.361356 8489 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:45:14.362221 8522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:14.362419 8489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:14.362552 8489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "30f615e293c844188e833853a53a4c95"
format_stamp: "Formatted at 2026-08-08 00:45:14 on dist-test-slave-kvfs"
server_key: "02e06c6fed8eb181c3fb422ff212900a"
server_key_iv: "960603964296e2d568d3ee2d916fcc4f"
server_key_version: "encryptionkey@0"
I20260808 00:45:14.362643 8489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:14.389180 8489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:14.392123 8489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:14.392330 8489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:14.397717 8489 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:41745
I20260808 00:45:14.397765 8595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:41745 every 8 connection(s)
I20260808 00:45:14.398255 8489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:14.399467 8599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:14.402127 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 8489
I20260808 00:45:14.402256 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:45:14.402506 20901 external_mini_cluster.cc:1428] Setting key 28ca4645c7a49babe9d16805d838ba20
I20260808 00:45:14.403048 8599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95: Bootstrap starting.
I20260808 00:45:14.403977 8599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:14.404475 8599 log.cc:824] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:14.405488 8599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95: No bootstrap required, opened a new log
I20260808 00:45:14.408538 8599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f615e293c844188e833853a53a4c95" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41745 } }
I20260808 00:45:14.408775 8599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:14.408809 8599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f615e293c844188e833853a53a4c95, State: Initialized, Role: FOLLOWER
I20260808 00:45:14.409009 8599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f615e293c844188e833853a53a4c95" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41745 } }
I20260808 00:45:14.409127 8599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:45:14.409175 8599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:45:14.409300 8599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:14.410446 8599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f615e293c844188e833853a53a4c95" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41745 } }
I20260808 00:45:14.410621 8599 leader_election.cc:302] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30f615e293c844188e833853a53a4c95; no voters:
I20260808 00:45:14.410892 8599 leader_election.cc:288] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:45:14.410939 8605 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:14.411162 8605 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [term 1 LEADER]: Becoming Leader. State: Replica: 30f615e293c844188e833853a53a4c95, State: Running, Role: LEADER
Aug 08 00:45:14 dist-test-slave-kvfs krb5kdc[8472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:14.411264 8599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:14.411334 8605 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f615e293c844188e833853a53a4c95" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41745 } }
I20260808 00:45:14.411829 8606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30f615e293c844188e833853a53a4c95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f615e293c844188e833853a53a4c95" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41745 } } }
I20260808 00:45:14.411933 8606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:14.411814 8605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30f615e293c844188e833853a53a4c95. Latest consensus state: current_term: 1 leader_uuid: "30f615e293c844188e833853a53a4c95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30f615e293c844188e833853a53a4c95" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 41745 } } }
I20260808 00:45:14.412372 8605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:14.412724 8617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:14.413501 8617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:45:14.415783 8617 catalog_manager.cc:1347] Generated new cluster ID: 7dba84ba26804d6a8874f02814be7a4d
I20260808 00:45:14.415850 8617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:45:14.417073 8603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:14.403212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49434 (local address 127.20.105.126:41745)
0808 00:45:14.403445 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:14.403456 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:14.403489 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:45:14.404130 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:14.404140 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:14.404204 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:14.404347 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:14.405353 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:14.406420 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:14.407690 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:14.408224 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:14.408316 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:14.411928 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:14.411933 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:14.411933 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:14.411948 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:14.414503 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:14.415008 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:14.415012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:14.415013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:14.415145 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:14.415437 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:14.415438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:14.415438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:14.415828 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:14.415996 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:14.416635 (+ 639us) server_negotiation.cc:299] Negotiation successful
0808 00:45:14.416799 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":115,"threads_started":1}
I20260808 00:45:14.418044 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:45:14.427096 8617 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:45:14.427960 8617 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:45:14.431511 8617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30f615e293c844188e833853a53a4c95: Generated new TSK 0
I20260808 00:45:14.431814 8617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:45:14.472793 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 8489
2026-08-08T00:45:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1705 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:45:14 dist-test-slave-kvfs krb5kdc[8639](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:45:14 dist-test-slave-kvfs krb5kdc[8639](info): set up 2 sockets
Aug 08 00:45:14 dist-test-slave-kvfs krb5kdc[8639](info): commencing operation
krb5kdc: starting...
W20260808 00:45:15.785308 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:45:15 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:45:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:45:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:45:15.952747 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40993
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:40993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:16.055809 8658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:16.056154 8658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:16.056198 8658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:16.057579 8658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:16.057662 8658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:16.057685 8658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:16.057708 8658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:16.057734 8658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:16.059491 8658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:40993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:16.059862 8658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:16.060195 8658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:16.066411 8667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:16.066391 8669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:16.066399 8666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:16.066598 8658 server_base.cc:1034] running on GCE node
I20260808 00:45:16.067101 8658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:16.067500 8658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:16.068699 8658 hybrid_clock.cc:648] HybridClock initialized: now 1786149916068677 us; error 46 us; skew 500 ppm
Aug 08 00:45:16 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:16.071863 8658 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:16.072530 8658 webserver.cc:460] Webserver started at http://127.20.105.126:39385/ using document root <none> and password file <none>
I20260808 00:45:16.072811 8658 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:16.072887 8658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:16.073092 8658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:16.074415 8658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "bf6858f8137c4ac5b218b6eeeade2ed4"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d7099846f6a6ae0f03a7dc22abb00999"
server_key_iv: "1984cced6cc52569bf409cc4ff176fdf"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.074856 8658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "bf6858f8137c4ac5b218b6eeeade2ed4"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d7099846f6a6ae0f03a7dc22abb00999"
server_key_iv: "1984cced6cc52569bf409cc4ff176fdf"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.076588 8658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 00:45:16.077666 8678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.078030 8658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:16.078169 8658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "bf6858f8137c4ac5b218b6eeeade2ed4"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d7099846f6a6ae0f03a7dc22abb00999"
server_key_iv: "1984cced6cc52569bf409cc4ff176fdf"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.078267 8658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:16.093956 8658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:16.096698 8658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:16.096911 8658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:16.102550 8658 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:40993
I20260808 00:45:16.102592 8759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:40993 every 8 connection(s)
I20260808 00:45:16.103089 8658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:16.104512 8760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:16.107566 8760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Bootstrap starting.
I20260808 00:45:16.108492 8760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:16.108814 8760 log.cc:824] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:16.109663 8760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: No bootstrap required, opened a new log
I20260808 00:45:16.111697 8760 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:16.111882 8760 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:16.111913 8760 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6858f8137c4ac5b218b6eeeade2ed4, State: Initialized, Role: FOLLOWER
I20260808 00:45:16.112157 8760 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:16.112251 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 8658
I20260808 00:45:16.112280 8760 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:45:16.112323 8760 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:45:16.112386 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:45:16.112406 8760 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:16.112658 20901 external_mini_cluster.cc:1428] Setting key fd23b26cdc8c8425298df608819a23b3
I20260808 00:45:16.113859 8760 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:16.114012 8760 leader_election.cc:302] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf6858f8137c4ac5b218b6eeeade2ed4; no voters:
I20260808 00:45:16.114234 8760 leader_election.cc:288] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:45:16.114279 8768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:16.114449 8768 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 1 LEADER]: Becoming Leader. State: Replica: bf6858f8137c4ac5b218b6eeeade2ed4, State: Running, Role: LEADER
I20260808 00:45:16.114588 8768 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:16.114634 8760 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:16.116392 8771 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf6858f8137c4ac5b218b6eeeade2ed4. Latest consensus state: current_term: 1 leader_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } } }
I20260808 00:45:16.116423 8770 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } } }
I20260808 00:45:16.116592 8770 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:16.116849 8771 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:16.117007 8789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:16.117642 8789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:45:16.119800 8789 catalog_manager.cc:1347] Generated new cluster ID: 28e1cd8651ba41568fceee82f69a7d0a
I20260808 00:45:16.119869 8789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:45:16 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:16.126678 8789 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:45:16.127445 8789 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:45:16.131176 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.113359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.20.105.126:40993)
0808 00:45:16.113590 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.113600 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.113633 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:45:16.113753 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.113760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.113812 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.113973 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:16.114955 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.116552 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.117656 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.118210 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.118365 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.126670 (+ 8305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:16.126678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:16.126678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:16.126698 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:16.129176 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.129668 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.129672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.129672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.129802 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.130025 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.130026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.130026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.130380 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:16.130579 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.130676 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.130863 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
I20260808 00:45:16.134505 8789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Generated new TSK 0
I20260808 00:45:16.134727 8789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:45:16.171993 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40993
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:16.275709 8797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:16.275986 8797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:16.276013 8797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:16.277345 8797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:16.277402 8797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:16.277432 8797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:45:16.279035 8797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:16.279295 8797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:16.279553 8797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:16.286490 8809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:16.286638 8797 server_base.cc:1034] running on GCE node
W20260808 00:45:16.286742 8814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:16.286491 8806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:16.287057 8797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:16.287547 8797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:16.288774 8797 hybrid_clock.cc:648] HybridClock initialized: now 1786149916288759 us; error 51 us; skew 500 ppm
Aug 08 00:45:16 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:16.292106 8797 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:45:16.292709 8797 webserver.cc:460] Webserver started at http://127.20.105.65:36103/ using document root <none> and password file <none>
I20260808 00:45:16.292989 8797 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:16.293064 8797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:16.293236 8797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:16.294584 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "95afc8cb08544ef6bc208b962ac0a567"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "23af1199b1210afb58861b5c2b13a066"
server_key_iv: "6ddd19439b3517ce7baf62ddd0f5bf21"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.295028 8797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "95afc8cb08544ef6bc208b962ac0a567"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "23af1199b1210afb58861b5c2b13a066"
server_key_iv: "6ddd19439b3517ce7baf62ddd0f5bf21"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.296880 8797 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 00:45:16.297848 8824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.298128 8797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:16.298269 8797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "95afc8cb08544ef6bc208b962ac0a567"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "23af1199b1210afb58861b5c2b13a066"
server_key_iv: "6ddd19439b3517ce7baf62ddd0f5bf21"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.298372 8797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:16.315631 8797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:16.318856 8797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:16.319067 8797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:16.319420 8797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:16.319844 8797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:16.319909 8797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.319969 8797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:16.319995 8797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.328198 8797 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:35589
I20260808 00:45:16.328233 8994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:35589 every 8 connection(s)
I20260808 00:45:16.328719 8797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:45:16.331020 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 8797
I20260808 00:45:16.331174 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:45:16.331399 20901 external_mini_cluster.cc:1428] Setting key 09853bb39b0b20d172ac317601398a4c
Aug 08 00:45:16 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:16.341586 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.329679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:39913 (local address 127.20.105.126:40993)
0808 00:45:16.329780 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.329782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.329996 (+ 214us) server_negotiation.cc:407] Connection header received
0808 00:45:16.330621 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.330623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.330636 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.330714 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:16.332280 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.333119 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.334349 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.334694 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.334779 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.337660 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:16.337662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:16.337662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:16.337674 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:16.339649 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.340299 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.340301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.340302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.340385 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.340721 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.340722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.340723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.340876 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:16.340964 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.341320 (+ 356us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.341442 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 00:45:16.341766 8997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.329630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:39913)
0808 00:45:16.329828 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:16.329883 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:16.330519 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:16.330859 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:16.330864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:16.331826 (+ 962us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:16.332159 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.332165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.333250 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.333251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.334233 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.334236 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.334810 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.334811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.334960 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.335684 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:16.335695 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:16.337536 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:16.339805 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:16.339820 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:16.339823 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:16.340211 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:16.340485 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:16.340486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:16.340486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:16.340615 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:16.340950 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:16.340995 (+ 45us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:16.341158 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 00:45:16.341278 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 00:45:16.341481 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":101,"threads_started":1}
I20260808 00:45:16.342641 8995 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40993
I20260808 00:45:16.342792 8995 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:16.343061 8995 heartbeater.cc:511] Master 127.20.105.126:40993 requested a full tablet report, sending...
I20260808 00:45:16.343865 8702 ts_manager.cc:194] Registered new tserver with Master: 95afc8cb08544ef6bc208b962ac0a567 (127.20.105.65:35589)
I20260808 00:45:16.344679 8702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:39913
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:45:16.369042 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40993
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:16.476464 9004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:16.476742 9004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:16.476770 9004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:16.478018 9004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:16.478058 9004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:16.478086 9004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:45:16.479681 9004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:16.479941 9004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:16.480232 9004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:16.486500 9017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:16.486493 9015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:16.486744 9004 server_base.cc:1034] running on GCE node
W20260808 00:45:16.486493 9014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:16.487170 9004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:16.487568 9004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:16.488808 9004 hybrid_clock.cc:648] HybridClock initialized: now 1786149916488791 us; error 28 us; skew 500 ppm
Aug 08 00:45:16 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:16.491918 9004 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:45:16.492544 9004 webserver.cc:460] Webserver started at http://127.20.105.66:35589/ using document root <none> and password file <none>
I20260808 00:45:16.492851 9004 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:16.492923 9004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:16.493116 9004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:16.494364 9004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "90f0fdb865484d02bfa134d61875c14a"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "5e22826aebd4c5829a795cb059fd35ef"
server_key_iv: "7138f1504c6a68702606313503f32bae"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.494736 9004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "90f0fdb865484d02bfa134d61875c14a"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "5e22826aebd4c5829a795cb059fd35ef"
server_key_iv: "7138f1504c6a68702606313503f32bae"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.496321 9004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:45:16.497210 9025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.497488 9004 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:16.497577 9004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "90f0fdb865484d02bfa134d61875c14a"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "5e22826aebd4c5829a795cb059fd35ef"
server_key_iv: "7138f1504c6a68702606313503f32bae"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.497640 9004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:16.505671 9004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:16.508224 9004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:16.508400 9004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:16.508682 9004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:16.509070 9004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:16.509109 9004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.509155 9004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:16.509182 9004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.516760 9004 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:39321
I20260808 00:45:16.516813 9190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:39321 every 8 connection(s)
I20260808 00:45:16.517228 9004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:45:16.518210 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 9004
I20260808 00:45:16.518354 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:45:16.518615 20901 external_mini_cluster.cc:1428] Setting key 7408a840c1feefa8b053769a73d71fc5
Aug 08 00:45:16 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:16.530303 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.518124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:37447 (local address 127.20.105.126:40993)
0808 00:45:16.518221 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.518224 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.518468 (+ 244us) server_negotiation.cc:407] Connection header received
0808 00:45:16.519210 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.519212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.519223 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.519285 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:16.520861 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.521418 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.522813 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.523156 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.523234 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.526304 (+ 3070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:16.526306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:16.526307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:16.526318 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:16.528351 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.528996 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.528998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.528999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.529090 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.529383 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.529384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.529385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.529573 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:16.529658 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.530045 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.530162 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:45:16.530534 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.518078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:37447)
0808 00:45:16.518301 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:16.518369 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:16.519104 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:16.519353 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:16.519357 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:16.520366 (+ 1009us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:16.520749 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.520757 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.521518 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.521519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.522713 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.522716 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.523253 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.523254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.523384 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.524153 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:16.524173 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:16.526186 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:16.528481 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:16.528489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:16.528491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:16.528886 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:16.529172 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:16.529173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:16.529173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:16.529305 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:16.529681 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:16.529690 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:16.529877 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 00:45:16.529997 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 00:45:16.530168 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":116,"threads_started":1}
I20260808 00:45:16.531272 9191 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40993
I20260808 00:45:16.531436 9191 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:16.531700 9191 heartbeater.cc:511] Master 127.20.105.126:40993 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
I20260808 00:45:16.532335 8702 ts_manager.cc:194] Registered new tserver with Master: 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321)
I20260808 00:45:16.532851 8702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:37447
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:45:16.555769 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40993
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:16.661975 9199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:16.662308 9199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:16.662341 9199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:16.663775 9199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:16.663856 9199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:16.663893 9199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:45:16.665745 9199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:16.666127 9199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:16.666450 9199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:16.672821 9205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:16.672770 9208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:16.673126 9199 server_base.cc:1034] running on GCE node
W20260808 00:45:16.672771 9204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:16.673470 9199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:16.673935 9199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:16.675155 9199 hybrid_clock.cc:648] HybridClock initialized: now 1786149916675153 us; error 52 us; skew 500 ppm
Aug 08 00:45:16 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:16.678846 9199 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:45:16.679481 9199 webserver.cc:460] Webserver started at http://127.20.105.67:40603/ using document root <none> and password file <none>
I20260808 00:45:16.679807 9199 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:16.679880 9199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:16.680071 9199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:16.681931 9199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "048133867c124634bf30e6520e8d282f"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d778a3f44dbdcb7bc70c378fa5f936e9"
server_key_iv: "8e637c1023a6d305f2cc2ef8782f3d61"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.682505 9199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "048133867c124634bf30e6520e8d282f"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d778a3f44dbdcb7bc70c378fa5f936e9"
server_key_iv: "8e637c1023a6d305f2cc2ef8782f3d61"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.684612 9199 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 00:45:16.685798 9220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.686314 9199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:16.686467 9199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "048133867c124634bf30e6520e8d282f"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d778a3f44dbdcb7bc70c378fa5f936e9"
server_key_iv: "8e637c1023a6d305f2cc2ef8782f3d61"
server_key_version: "encryptionkey@0"
I20260808 00:45:16.686600 9199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:16.711447 9199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:16.714226 9199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:16.714376 9199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:16.714658 9199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:16.714999 9199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:16.715091 9199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.715143 9199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:16.715179 9199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:16.722975 9199 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:36287
I20260808 00:45:16.723080 9377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:36287 every 8 connection(s)
I20260808 00:45:16.723518 9199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:45:16.724216 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 9199
I20260808 00:45:16.724375 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:45:16.724676 20901 external_mini_cluster.cc:1428] Setting key fd5289de6797e151ed261da58fd31cc3
Aug 08 00:45:16 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149916, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:16.735062 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.724661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:37227 (local address 127.20.105.126:40993)
0808 00:45:16.724767 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.724770 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.725031 (+ 261us) server_negotiation.cc:407] Connection header received
0808 00:45:16.725875 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.725877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.725893 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.725960 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:16.727178 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.727809 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.729035 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.729261 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.729329 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.731985 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:16.731986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:16.731987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:16.731995 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:16.733663 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.734116 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.734117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.734117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.734181 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.734415 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.734415 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.734416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.734543 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:16.734601 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.734887 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.734963 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 00:45:16.735347 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.724634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:37227)
0808 00:45:16.724875 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:16.724928 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:16.725689 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:16.726081 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:16.726086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:16.726796 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:16.727051 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.727056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.727910 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.727911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.728935 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.728938 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.729371 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.729372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.729506 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.730182 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:16.730193 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:16.731879 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:16.733750 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:16.733757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:16.733760 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:16.734028 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:16.734242 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:16.734242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:16.734243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:16.734359 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:16.734601 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:16.734606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:16.734760 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 00:45:16.734855 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 00:45:16.735006 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":133,"threads_started":1}
I20260808 00:45:16.736102 9382 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40993
I20260808 00:45:16.736241 9382 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:16.736500 9382 heartbeater.cc:511] Master 127.20.105.126:40993 requested a full tablet report, sending...
I20260808 00:45:16.737002 8702 ts_manager.cc:194] Registered new tserver with Master: 048133867c124634bf30e6520e8d282f (127.20.105.67:36287)
I20260808 00:45:16.737449 8702 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:37227
I20260808 00:45:16.748492 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:45:16.758553 8767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.750765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49190 (local address 127.20.105.126:40993)
0808 00:45:16.750846 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.750849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.750989 (+ 140us) server_negotiation.cc:407] Connection header received
0808 00:45:16.751110 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.751111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.751124 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.751201 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:16.751895 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.752540 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.753357 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.753585 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.753670 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.755117 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:16.755119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:16.755119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:16.755128 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:16.757068 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.757526 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.757528 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.757529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.757611 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:16.757835 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:16.757836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:16.757836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:16.758049 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:16.758174 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.758323 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.758416 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:45:16.760980 8702 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:45:16.763298 8702 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:45:16.771442 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.767149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39321 (local address 127.0.0.1:36802)
0808 00:45:16.767628 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:16.767639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:16.767701 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:16.767947 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:16.767948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:16.767950 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:16.767989 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.767990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.768923 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.768925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.770016 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.770020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.771055 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.771056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.771145 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.771161 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:45:16.771246 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 00:45:16.771328 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260808 00:45:16.771461 9404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.767335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36802 (local address 127.20.105.66:39321)
0808 00:45:16.767614 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.767618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.767785 (+ 167us) server_negotiation.cc:407] Connection header received
0808 00:45:16.767836 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.767839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.767858 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.767942 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:16.768068 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.768811 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.770132 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.770947 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.771036 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.771108 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.771285 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.771343 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":112,"threads_started":1}
I20260808 00:45:16.771688 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.766925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:36287 (local address 127.0.0.1:46366)
0808 00:45:16.767142 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:16.767153 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:16.767224 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:16.767720 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:16.767721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:16.767724 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:16.767794 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.767796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.769317 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.769318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.770517 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.770521 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.771433 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.771434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.771500 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.771505 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:45:16.771555 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:45:16.771610 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260808 00:45:16.771955 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.767124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46366 (local address 127.20.105.67:36287)
0808 00:45:16.767412 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.767418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.767431 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:45:16.767480 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.767483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.767500 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.767617 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:16.767894 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.769172 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.770634 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.771284 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.771343 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.771398 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.771810 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.771863 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":123,"threads_started":1}
I20260808 00:45:16.772511 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.766684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:35589 (local address 127.0.0.1:37278)
0808 00:45:16.766842 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:16.766896 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:16.767005 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:16.767639 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:16.767643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:16.767646 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:16.767774 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.767776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.769305 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.769306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.770535 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.770537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.772233 (+ 1696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.772234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.772311 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.772315 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:45:16.772380 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 00:45:16.772430 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":76,"thread_start_us":62,"threads_started":1}
I20260808 00:45:16.772583 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.766911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37278 (local address 127.20.105.65:35589)
0808 00:45:16.767378 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.767385 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.767397 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:45:16.767449 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.767452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.767472 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.767886 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:16.767994 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.769164 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.770639 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.771448 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.772205 (+ 757us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.772281 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.772420 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.772486 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":107,"threads_started":1}
I20260808 00:45:16.772500 9093 tablet_service.cc:1467] Processing CreateTablet for tablet a142699473b94be4902d56ca3c25c6e7 (DEFAULT_TABLE table=test-table [id=bbacb557c6dc4ac4890e87a7aaa90509]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:16.772985 9093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a142699473b94be4902d56ca3c25c6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:16.772994 9281 tablet_service.cc:1467] Processing CreateTablet for tablet a142699473b94be4902d56ca3c25c6e7 (DEFAULT_TABLE table=test-table [id=bbacb557c6dc4ac4890e87a7aaa90509]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:16.773303 9281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a142699473b94be4902d56ca3c25c6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:16.776473 9405 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: Bootstrap starting.
I20260808 00:45:16.776693 9406 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Bootstrap starting.
I20260808 00:45:16.777216 8893 tablet_service.cc:1467] Processing CreateTablet for tablet a142699473b94be4902d56ca3c25c6e7 (DEFAULT_TABLE table=test-table [id=bbacb557c6dc4ac4890e87a7aaa90509]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:16.777451 9406 tablet_bootstrap.cc:654] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:16.777452 9405 tablet_bootstrap.cc:654] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:16.777640 8893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a142699473b94be4902d56ca3c25c6e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:16.778246 9406 log.cc:824] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:16.778261 9405 log.cc:824] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:16.779148 9406 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: No bootstrap required, opened a new log
I20260808 00:45:16.779213 9406 ts_tablet_manager.cc:1397] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:45:16.779693 9405 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: No bootstrap required, opened a new log
I20260808 00:45:16.779773 9405 ts_tablet_manager.cc:1397] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:45:16.781183 9409 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Bootstrap starting.
I20260808 00:45:16.781467 9406 raft_consensus.cc:348] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.781682 9406 raft_consensus.cc:374] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:16.781729 9406 raft_consensus.cc:729] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048133867c124634bf30e6520e8d282f, State: Initialized, Role: FOLLOWER
I20260808 00:45:16.781919 9406 consensus_queue.cc:260] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.782126 9409 tablet_bootstrap.cc:654] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:16.782029 9405 raft_consensus.cc:348] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.782225 9405 raft_consensus.cc:374] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:16.782267 9406 ts_tablet_manager.cc:1428] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:45:16.782275 9405 raft_consensus.cc:729] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f0fdb865484d02bfa134d61875c14a, State: Initialized, Role: FOLLOWER
I20260808 00:45:16.782398 9405 consensus_queue.cc:260] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.782514 9382 heartbeater.cc:503] Master 127.20.105.126:40993 was elected leader, sending a full tablet report...
I20260808 00:45:16.782723 9405 ts_tablet_manager.cc:1428] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:45:16.782876 9191 heartbeater.cc:503] Master 127.20.105.126:40993 was elected leader, sending a full tablet report...
I20260808 00:45:16.783093 9409 log.cc:824] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:16.784092 9409 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: No bootstrap required, opened a new log
I20260808 00:45:16.784178 9409 ts_tablet_manager.cc:1397] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:45:16.785740 9409 raft_consensus.cc:348] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.785889 9409 raft_consensus.cc:374] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:16.785912 9409 raft_consensus.cc:729] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95afc8cb08544ef6bc208b962ac0a567, State: Initialized, Role: FOLLOWER
I20260808 00:45:16.786044 9409 consensus_queue.cc:260] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.786367 9409 ts_tablet_manager.cc:1428] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:45:16.786397 8995 heartbeater.cc:503] Master 127.20.105.126:40993 was elected leader, sending a full tablet report...
W20260808 00:45:16.829715 8996 tablet.cc:2369] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:45:16.948196 9415 raft_consensus.cc:482] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:16.948391 9415 raft_consensus.cc:504] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.948841 9415 leader_election.cc:288] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 048133867c124634bf30e6520e8d282f (127.20.105.67:36287), 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321)
I20260808 00:45:16.953553 9404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.949669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:33239 (local address 127.20.105.66:39321)
0808 00:45:16.949745 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.949747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.949799 (+ 52us) server_negotiation.cc:407] Connection header received
0808 00:45:16.949957 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.949959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.949977 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.950038 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:16.950332 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.951048 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.952496 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.953120 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.953205 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.953217 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.953408 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.953453 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:45:16.953548 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.949295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:46129 (local address 127.20.105.67:36287)
0808 00:45:16.949368 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.949371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.949475 (+ 104us) server_negotiation.cc:407] Connection header received
0808 00:45:16.949605 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.949608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.949623 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.949665 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:16.949990 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.950824 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.952501 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.953120 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.953205 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.953217 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:16.953412 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 00:45:16.953453 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:45:16.953630 9417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.949476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39321 (local address 127.20.105.65:33239)
0808 00:45:16.949705 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:16.949717 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:16.949830 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:16.950138 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:16.950139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:16.950141 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:16.950244 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.950246 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.951147 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.951148 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.952390 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.952394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.953222 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.953222 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.953316 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.953322 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:45:16.953387 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 00:45:16.953454 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":76,"thread_start_us":84,"threads_started":1}
I20260808 00:45:16.953630 9416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.949168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:36287 (local address 127.20.105.65:46129)
0808 00:45:16.949369 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:16.949384 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:16.949480 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:16.949779 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:16.949780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:16.949783 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:16.949913 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.949916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.950946 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.950947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.952390 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:16.952394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.953220 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:16.953221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:16.953316 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:16.953321 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:45:16.953387 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:45:16.953453 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":9,"thread_start_us":131,"threads_started":1}
I20260808 00:45:16.954207 9121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a142699473b94be4902d56ca3c25c6e7" candidate_uuid: "95afc8cb08544ef6bc208b962ac0a567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f0fdb865484d02bfa134d61875c14a" is_pre_election: true
I20260808 00:45:16.954468 9121 raft_consensus.cc:2454] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95afc8cb08544ef6bc208b962ac0a567 in term 0.
I20260808 00:45:16.954423 9314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a142699473b94be4902d56ca3c25c6e7" candidate_uuid: "95afc8cb08544ef6bc208b962ac0a567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048133867c124634bf30e6520e8d282f" is_pre_election: true
I20260808 00:45:16.954572 9314 raft_consensus.cc:2454] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95afc8cb08544ef6bc208b962ac0a567 in term 0.
I20260808 00:45:16.954797 8830 leader_election.cc:302] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90f0fdb865484d02bfa134d61875c14a, 95afc8cb08544ef6bc208b962ac0a567; no voters:
I20260808 00:45:16.954949 9415 raft_consensus.cc:2781] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:45:16.955016 9415 raft_consensus.cc:482] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:45:16.955044 9415 raft_consensus.cc:3037] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:16.955960 9415 raft_consensus.cc:504] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.956169 9415 leader_election.cc:288] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 1 election: Requested vote from peers 048133867c124634bf30e6520e8d282f (127.20.105.67:36287), 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321)
I20260808 00:45:16.956385 9314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a142699473b94be4902d56ca3c25c6e7" candidate_uuid: "95afc8cb08544ef6bc208b962ac0a567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048133867c124634bf30e6520e8d282f"
I20260808 00:45:16.956421 9121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a142699473b94be4902d56ca3c25c6e7" candidate_uuid: "95afc8cb08544ef6bc208b962ac0a567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f0fdb865484d02bfa134d61875c14a"
I20260808 00:45:16.956488 9314 raft_consensus.cc:3037] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:16.956552 9121 raft_consensus.cc:3037] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:16.957406 9314 raft_consensus.cc:2454] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95afc8cb08544ef6bc208b962ac0a567 in term 1.
I20260808 00:45:16.957598 9121 raft_consensus.cc:2454] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95afc8cb08544ef6bc208b962ac0a567 in term 1.
I20260808 00:45:16.957657 8830 leader_election.cc:302] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048133867c124634bf30e6520e8d282f, 95afc8cb08544ef6bc208b962ac0a567; no voters:
I20260808 00:45:16.957769 9415 raft_consensus.cc:2781] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:16.958025 9415 raft_consensus.cc:686] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 LEADER]: Becoming Leader. State: Replica: 95afc8cb08544ef6bc208b962ac0a567, State: Running, Role: LEADER
I20260808 00:45:16.958186 9415 consensus_queue.cc:237] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:16.959532 8699 catalog_manager.cc:5495] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95afc8cb08544ef6bc208b962ac0a567 (127.20.105.65). New cstate: current_term: 1 leader_uuid: "95afc8cb08544ef6bc208b962ac0a567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } health_report { overall_health: UNKNOWN } } }
W20260808 00:45:16.974440 9383 tablet.cc:2369] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:45:17.001214 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:16.998240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37282 (local address 127.20.105.65:35589)
0808 00:45:16.998319 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:16.998321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:16.998331 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:45:16.998389 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:16.998391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:16.998416 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:16.998508 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:45:16.998687 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:16.999213 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.000096 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.000325 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.000417 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.000603 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:45:17.000906 (+ 303us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:45:17.000989 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.001045 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.001073 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 00:45:17.003861 9314 raft_consensus.cc:1261] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 1 FOLLOWER]: Refusing update from remote peer 95afc8cb08544ef6bc208b962ac0a567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:17.003887 9121 raft_consensus.cc:1261] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 1 FOLLOWER]: Refusing update from remote peer 95afc8cb08544ef6bc208b962ac0a567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:17.004410 9419 consensus_queue.cc:1035] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:17.004602 9415 consensus_queue.cc:1035] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:17.011204 8702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:17.011375 8702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:17.012452 8702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Sending DeleteTablet for 3 replicas of tablet a142699473b94be4902d56ca3c25c6e7
I20260808 00:45:17.012954 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 8797
I20260808 00:45:17.012989 8893 tablet_service.cc:1514] Processing DeleteTablet for tablet a142699473b94be4902d56ca3c25c6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37278
I20260808 00:45:17.013123 9281 tablet_service.cc:1514] Processing DeleteTablet for tablet a142699473b94be4902d56ca3c25c6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:46366
I20260808 00:45:17.016377 9093 tablet_service.cc:1514] Processing DeleteTablet for tablet a142699473b94be4902d56ca3c25c6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:36802
I20260808 00:45:17.023845 9437 tablet_replica.cc:330] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: stopping tablet replica
I20260808 00:45:17.023845 9436 tablet_replica.cc:330] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: stopping tablet replica
I20260808 00:45:17.024065 9436 raft_consensus.cc:2229] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:17.024091 9437 raft_consensus.cc:2229] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:17.024236 9437 raft_consensus.cc:2258] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:17.024236 9436 raft_consensus.cc:2258] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:17.024885 9437 ts_tablet_manager.cc:1905] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:17.024885 9436 ts_tablet_manager.cc:1905] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:17.027076 9436 ts_tablet_manager.cc:1918] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:17.027107 9437 ts_tablet_manager.cc:1918] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:17.027153 9436 log.cc:1196] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/a142699473b94be4902d56ca3c25c6e7
I20260808 00:45:17.027196 9437 log.cc:1196] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/a142699473b94be4902d56ca3c25c6e7
I20260808 00:45:17.027400 9436 ts_tablet_manager.cc:1939] T a142699473b94be4902d56ca3c25c6e7 P 048133867c124634bf30e6520e8d282f: Deleting consensus metadata
I20260808 00:45:17.027489 9437 ts_tablet_manager.cc:1939] T a142699473b94be4902d56ca3c25c6e7 P 90f0fdb865484d02bfa134d61875c14a: Deleting consensus metadata
I20260808 00:45:17.028401 8682 catalog_manager.cc:4841] TS 048133867c124634bf30e6520e8d282f (127.20.105.67:36287): tablet a142699473b94be4902d56ca3c25c6e7 (table test-table [id=bbacb557c6dc4ac4890e87a7aaa90509]) successfully deleted
I20260808 00:45:17.028564 8682 catalog_manager.cc:4841] TS 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321): tablet a142699473b94be4902d56ca3c25c6e7 (table test-table [id=bbacb557c6dc4ac4890e87a7aaa90509]) successfully deleted
W20260808 00:45:17.033116 8682 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:35589 (error 108)
I20260808 00:45:17.033558 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 9004
I20260808 00:45:17.036823 9402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.033763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:35589 (local address 127.0.0.1:37296)
0808 00:45:17.036683 (+ 2920us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.036761 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:35589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2878}
W20260808 00:45:17.040552 8682 catalog_manager.cc:4584] TS 95afc8cb08544ef6bc208b962ac0a567 (127.20.105.65:35589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a142699473b94be4902d56ca3c25c6e7: Network error: Client connection negotiation failed: client connection to 127.20.105.65:35589: connect: Connection refused (error 111)
I20260808 00:45:17.053025 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 9199
I20260808 00:45:17.065992 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 8658
I20260808 00:45:17.081564 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40993
--webserver_interface=127.20.105.126
--webserver_port=39385
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:40993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:17.182157 9441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:17.182477 9441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:17.182518 9441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:17.183823 9441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:17.183899 9441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:17.183924 9441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:17.183947 9441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:17.183979 9441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:17.185712 9441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:40993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=39385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9441
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:17.186093 9441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:17.186393 9441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:17.198657 9450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:17.198905 9441 server_base.cc:1034] running on GCE node
W20260808 00:45:17.198657 9453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:17.198668 9456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:17.199378 9441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:17.199831 9441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:17.201022 9441 hybrid_clock.cc:648] HybridClock initialized: now 1786149917200996 us; error 55 us; skew 500 ppm
Aug 08 00:45:17 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:17.204144 9441 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:17.204689 9441 webserver.cc:460] Webserver started at http://127.20.105.126:39385/ using document root <none> and password file <none>
I20260808 00:45:17.204939 9441 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:17.205000 9441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:17.206751 9441 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:45:17.207731 9477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:17.207934 9441 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:17.208093 9441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "bf6858f8137c4ac5b218b6eeeade2ed4"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d7099846f6a6ae0f03a7dc22abb00999"
server_key_iv: "1984cced6cc52569bf409cc4ff176fdf"
server_key_version: "encryptionkey@0"
I20260808 00:45:17.208670 9441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:17.235867 9441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:17.239012 9441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:17.239171 9441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:17.246047 9441 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:40993
I20260808 00:45:17.246219 9552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:40993 every 8 connection(s)
I20260808 00:45:17.246754 9441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:17.249843 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 9441
I20260808 00:45:17.250255 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:35589
--local_ip_for_outbound_sockets=127.20.105.65
--tserver_master_addrs=127.20.105.126:40993
--webserver_port=36103
--webserver_interface=127.20.105.65
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:17.250576 9553 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:45:17.251878 9553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Bootstrap starting.
I20260808 00:45:17.265801 9553 log.cc:824] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:17.269698 9553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:45:17.270154 9553 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Bootstrap complete.
I20260808 00:45:17.274161 9553 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:17.274566 9553 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6858f8137c4ac5b218b6eeeade2ed4, State: Initialized, Role: FOLLOWER
I20260808 00:45:17.274768 9553 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:17.274840 9553 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:45:17.274899 9553 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:45:17.274952 9553 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:45:17.276749 9553 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:17.276891 9553 leader_election.cc:302] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf6858f8137c4ac5b218b6eeeade2ed4; no voters:
I20260808 00:45:17.277128 9553 leader_election.cc:288] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:45:17.277233 9558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:45:17.277439 9558 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 2 LEADER]: Becoming Leader. State: Replica: bf6858f8137c4ac5b218b6eeeade2ed4, State: Running, Role: LEADER
I20260808 00:45:17.277468 9553 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:17.277585 9558 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:17.277891 9559 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } } }
I20260808 00:45:17.277983 9559 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:17.278616 9564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:17.279333 9560 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf6858f8137c4ac5b218b6eeeade2ed4. Latest consensus state: current_term: 2 leader_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } } }
I20260808 00:45:17.279417 9560 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:17.279760 9564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:45:17.279903 9564 catalog_manager.cc:1259] Loaded cluster ID: 28e1cd8651ba41568fceee82f69a7d0a
I20260808 00:45:17.279951 9564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:45:17.280422 9564 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 00:45:17.283367 9564 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 00:45:17.291179 9564 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:45:17.291553 9564 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Loaded TSK: 0
I20260808 00:45:17.291893 9564 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:45:17.375406 9556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:17.375708 9556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:17.375741 9556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:17.377488 9556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:17.377568 9556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:17.377614 9556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:45:17.380132 9556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:35589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=36103
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:17.380513 9556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:17.380848 9556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:17.388463 9591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:17.388494 9593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:17.388569 9596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:17.389007 9556 server_base.cc:1034] running on GCE node
I20260808 00:45:17.389307 9556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:17.389678 9556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:17.390894 9556 hybrid_clock.cc:648] HybridClock initialized: now 1786149917390868 us; error 46 us; skew 500 ppm
Aug 08 00:45:17 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:17.394063 9556 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:45:17.394704 9556 webserver.cc:460] Webserver started at http://127.20.105.65:36103/ using document root <none> and password file <none>
I20260808 00:45:17.394987 9556 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:17.395093 9556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:17.396946 9556 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:45:17.397905 9612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:17.398102 9556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:17.398219 9556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "95afc8cb08544ef6bc208b962ac0a567"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "23af1199b1210afb58861b5c2b13a066"
server_key_iv: "6ddd19439b3517ce7baf62ddd0f5bf21"
server_key_version: "encryptionkey@0"
I20260808 00:45:17.398720 9556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:17.414181 9556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:17.417196 9556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:17.417373 9556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:17.417702 9556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:17.418406 9621 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:45:17.421044 9556 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:45:17.421156 9556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 00:45:17.421213 9556 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:45:17.422096 9556 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:45:17.422158 9556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:45:17.422355 9621 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Bootstrap starting.
I20260808 00:45:17.431367 9556 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:35589
I20260808 00:45:17.431527 9770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:35589 every 8 connection(s)
I20260808 00:45:17.431933 9556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:45:17.432646 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 9556
I20260808 00:45:17.433065 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:39321
--local_ip_for_outbound_sockets=127.20.105.66
--tserver_master_addrs=127.20.105.126:40993
--webserver_port=35589
--webserver_interface=127.20.105.66
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:17.438339 9621 log.cc:824] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:17.442662 9621 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:45:17.443198 9621 tablet_bootstrap.cc:492] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Bootstrap complete.
I20260808 00:45:17.444756 9621 ts_tablet_manager.cc:1397] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.010s
Aug 08 00:45:17 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:17.446883 9621 raft_consensus.cc:348] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:17.447279 9621 raft_consensus.cc:729] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95afc8cb08544ef6bc208b962ac0a567, State: Initialized, Role: FOLLOWER
I20260808 00:45:17.447512 9621 consensus_queue.cc:260] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } }
I20260808 00:45:17.447908 9621 ts_tablet_manager.cc:1428] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 00:45:17.453187 9775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.433015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:54137)
0808 00:45:17.436248 (+ 3233us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.436344 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:17.437500 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:17.438094 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:17.438100 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:17.438802 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:17.439170 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.439181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.441483 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.441485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.443063 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.443069 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.444266 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.444268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.444517 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.445346 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:17.445361 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:17.447569 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:17.450730 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:17.450741 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:17.450744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:17.451149 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:17.451559 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:17.451561 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:17.451561 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:17.451727 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:17.452372 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:17.452382 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:17.452577 (+ 195us) client_negotiation.cc:769] Sending connection context
0808 00:45:17.452708 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:45:17.452934 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3150,"thread_start_us":121,"threads_started":1}
I20260808 00:45:17.453208 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.433171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:54137 (local address 127.20.105.126:40993)
0808 00:45:17.437510 (+ 4339us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.437524 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.437605 (+ 81us) server_negotiation.cc:407] Connection header received
0808 00:45:17.437802 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.437809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.437868 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.438048 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:17.439393 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.441265 (+ 1872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.443204 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.443965 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.444123 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.447785 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:17.447792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:17.447792 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:17.447808 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:17.450624 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.451267 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.451272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.451272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.451428 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.451813 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.451814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.451815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.452264 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:17.452362 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.452806 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.452995 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4215,"thread_start_us":106,"threads_started":1}
I20260808 00:45:17.454394 9772 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40993
I20260808 00:45:17.454533 9772 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:17.454800 9772 heartbeater.cc:511] Master 127.20.105.126:40993 requested a full tablet report, sending...
I20260808 00:45:17.455900 9503 ts_manager.cc:194] Registered new tserver with Master: 95afc8cb08544ef6bc208b962ac0a567 (127.20.105.65:35589)
I20260808 00:45:17.457441 9503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:54137
I20260808 00:45:17.458776 9772 heartbeater.cc:503] Master 127.20.105.126:40993 was elected leader, sending a full tablet report...
Aug 08 00:45:17 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:45:17.469772 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.456847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:35589 (local address 127.0.0.1:37308)
0808 00:45:17.457435 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.457481 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:17.457793 (+ 312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:17.459113 (+ 1320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:17.459122 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:17.459638 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:17.459716 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.459720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.461151 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.461153 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.462061 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.462066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.462835 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.462837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.462920 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.463654 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:17.463660 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:17.465561 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:17.468209 (+ 2648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:17.468221 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:17.468223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:17.468525 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:17.468816 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:17.468817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:17.468818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:17.468899 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:17.469294 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:17.469305 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:17.469374 (+ 69us) client_negotiation.cc:769] Sending connection context
0808 00:45:17.469501 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 00:45:17.469665 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":137,"threads_started":1}
I20260808 00:45:17.469779 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.457193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.20.105.65:35589)
0808 00:45:17.457725 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.457730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.457743 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:45:17.458013 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.458015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.458871 (+ 856us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.458995 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:17.459806 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.461031 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.462179 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.462717 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.462805 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.465681 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:17.465697 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:17.465697 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:17.465709 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:17.468092 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.468627 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.468631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.468632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.468710 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.468975 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.468976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.468977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.469193 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:17.469283 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.469557 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.469664 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":109,"threads_started":1}
I20260808 00:45:17.470482 9678 tablet_service.cc:1514] Processing DeleteTablet for tablet a142699473b94be4902d56ca3c25c6e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37308
I20260808 00:45:17.471297 9784 tablet_replica.cc:330] stopping tablet replica
I20260808 00:45:17.471477 9784 raft_consensus.cc:2229] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:17.471594 9784 raft_consensus.cc:2258] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:17.472165 9784 ts_tablet_manager.cc:1905] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:17.474684 9784 ts_tablet_manager.cc:1918] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:17.474782 9784 log.cc:1196] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/a142699473b94be4902d56ca3c25c6e7
I20260808 00:45:17.475224 9784 ts_tablet_manager.cc:1939] T a142699473b94be4902d56ca3c25c6e7 P 95afc8cb08544ef6bc208b962ac0a567: Deleting consensus metadata
I20260808 00:45:17.476169 9484 catalog_manager.cc:4841] TS 95afc8cb08544ef6bc208b962ac0a567 (127.20.105.65:35589): tablet a142699473b94be4902d56ca3c25c6e7 (table test-table [id=bbacb557c6dc4ac4890e87a7aaa90509]) successfully deleted
W20260808 00:45:17.548355 9778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:17.548671 9778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:17.548702 9778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:17.550004 9778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:17.550045 9778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:17.550082 9778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:45:17.551779 9778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:39321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=35589
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:17.552163 9778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:17.552431 9778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:17.558881 9791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:17.559018 9790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:17.559276 9778 server_base.cc:1034] running on GCE node
W20260808 00:45:17.559049 9794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:17.559748 9778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:17.560181 9778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:17.561348 9778 hybrid_clock.cc:648] HybridClock initialized: now 1786149917561327 us; error 50 us; skew 500 ppm
Aug 08 00:45:17 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:17.564630 9778 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:45:17.565232 9778 webserver.cc:460] Webserver started at http://127.20.105.66:35589/ using document root <none> and password file <none>
I20260808 00:45:17.565547 9778 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:17.565615 9778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:17.567305 9778 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:17.568152 9802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:17.568377 9778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:17.568490 9778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "90f0fdb865484d02bfa134d61875c14a"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "5e22826aebd4c5829a795cb059fd35ef"
server_key_iv: "7138f1504c6a68702606313503f32bae"
server_key_version: "encryptionkey@0"
I20260808 00:45:17.568956 9778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:17.597132 9778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:17.599664 9778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:17.599857 9778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:17.600234 9778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:17.600637 9778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:17.600677 9778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:17.600705 9778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:17.600718 9778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:17.608539 9778 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:39321
I20260808 00:45:17.608596 9981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:39321 every 8 connection(s)
I20260808 00:45:17.609072 9778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:45:17.614939 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 9778
I20260808 00:45:17.615320 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:36287
--local_ip_for_outbound_sockets=127.20.105.67
--tserver_master_addrs=127.20.105.126:40993
--webserver_port=40603
--webserver_interface=127.20.105.67
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:45:17 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:17.622663 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.610248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:56843 (local address 127.20.105.126:40993)
0808 00:45:17.610349 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.610352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.610520 (+ 168us) server_negotiation.cc:407] Connection header received
0808 00:45:17.611128 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.611129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.611142 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.611199 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:17.612502 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.613195 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.614479 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.614837 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.614923 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.618405 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:17.618407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:17.618408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:17.618418 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:17.620771 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.621388 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.621389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.621389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.621451 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.621767 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.621768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.621768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.621903 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:17.621958 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.622441 (+ 483us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.622565 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 00:45:17.622957 9985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.610167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:56843)
0808 00:45:17.610368 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.610414 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:17.611026 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:17.611267 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:17.611272 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:17.612096 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:17.612389 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.612397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.613315 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.613317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.614379 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.614382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.614984 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.614986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.615215 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.616008 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:17.616067 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:17.618267 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:17.620881 (+ 2614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:17.620893 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:17.620897 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:17.621293 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:17.621516 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:17.621517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:17.621517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:17.621700 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:17.621969 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:17.621977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:17.622195 (+ 218us) client_negotiation.cc:769] Sending connection context
0808 00:45:17.622363 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 00:45:17.622633 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260808 00:45:17.623688 9983 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40993
I20260808 00:45:17.623837 9983 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:17.624136 9983 heartbeater.cc:511] Master 127.20.105.126:40993 requested a full tablet report, sending...
I20260808 00:45:17.624732 9503 ts_manager.cc:194] Registered new tserver with Master: 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321)
I20260808 00:45:17.625227 9503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:56843
W20260808 00:45:17.718854 9986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:17.719285 9986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:17.719316 9986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:17.720834 9986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:17.720890 9986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:17.720928 9986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:45:17.722625 9986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:36287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=40603
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:17.722899 9986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:17.723176 9986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:17.729434 9993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:17.729441 9986 server_base.cc:1034] running on GCE node
W20260808 00:45:17.729458 9997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:17.729454 9994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:17.729964 9986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:17.730297 9986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:17.731451 9986 hybrid_clock.cc:648] HybridClock initialized: now 1786149917731444 us; error 39 us; skew 500 ppm
Aug 08 00:45:17 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:17.734565 9986 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:45:17.735136 9986 webserver.cc:460] Webserver started at http://127.20.105.67:40603/ using document root <none> and password file <none>
I20260808 00:45:17.735389 9986 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:17.735495 9986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:17.737206 9986 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:17.737959 10006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:17.738178 9986 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:17.738298 9986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "048133867c124634bf30e6520e8d282f"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d778a3f44dbdcb7bc70c378fa5f936e9"
server_key_iv: "8e637c1023a6d305f2cc2ef8782f3d61"
server_key_version: "encryptionkey@0"
I20260808 00:45:17.738775 9986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:17.759886 9986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:17.762905 9986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:17.763137 9986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:17.763458 9986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:17.763876 9986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:17.763936 9986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:17.763988 9986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:17.764014 9986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:17.771410 9986 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:36287
I20260808 00:45:17.771458 10175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:36287 every 8 connection(s)
I20260808 00:45:17.771912 9986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:45:17.776338 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 9986
Aug 08 00:45:17 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149917, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:17.784978 9777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.772904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:58033 (local address 127.20.105.126:40993)
0808 00:45:17.773011 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.773014 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.773225 (+ 211us) server_negotiation.cc:407] Connection header received
0808 00:45:17.773926 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.773927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.773940 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.773996 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:17.775212 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.775776 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.777463 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.777846 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.777954 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.781251 (+ 3297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:17.781254 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:17.781254 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:17.781261 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:17.783040 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.783613 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.783615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.783616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.783681 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.783946 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.783947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.783947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.784192 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:17.784327 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.784756 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.784857 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 00:45:17.785454 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.772855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:58033)
0808 00:45:17.773068 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.773116 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:17.773810 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:17.774076 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:17.774081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:17.774856 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:17.775115 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.775120 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.775888 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.775889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.777337 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.777341 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.778025 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.778027 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.778216 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.778952 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:17.778965 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:17.781124 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:17.783134 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:17.783163 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:17.783167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:17.783527 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:17.783756 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:17.783757 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:17.783757 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:17.783876 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:17.784324 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:17.784333 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:17.784561 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 00:45:17.784709 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 00:45:17.784922 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":120,"threads_started":1}
I20260808 00:45:17.785722 10183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.776967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49196 (local address 127.20.105.126:40993)
0808 00:45:17.777152 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.777154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.777289 (+ 135us) server_negotiation.cc:407] Connection header received
0808 00:45:17.777364 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.777365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.777380 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.777461 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:17.778451 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.779253 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.780426 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.780686 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.780750 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.782318 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:17.782320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:17.782321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:17.782330 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:17.784062 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.784486 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.784486 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.784487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.784569 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:17.784834 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:17.784835 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:17.784835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:17.784964 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:17.785443 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.785492 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.785616 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":103,"threads_started":1}
I20260808 00:45:17.786177 10179 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40993
I20260808 00:45:17.786309 10179 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:17.786559 10179 heartbeater.cc:511] Master 127.20.105.126:40993 requested a full tablet report, sending...
I20260808 00:45:17.787123 9503 ts_manager.cc:194] Registered new tserver with Master: 048133867c124634bf30e6520e8d282f (127.20.105.67:36287)
I20260808 00:45:17.787540 9503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:58033
I20260808 00:45:17.797855 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:45:17.801615 10183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.798358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49206 (local address 127.20.105.126:40993)
0808 00:45:17.798466 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.798468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.798595 (+ 127us) server_negotiation.cc:407] Connection header received
0808 00:45:17.798718 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.798719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.798774 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.798876 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:45:17.799076 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.799643 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.800600 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.800818 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.800898 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.801131 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:45:17.801284 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:45:17.801420 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.801443 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.801500 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 00:45:17.802042 9503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:45:17.804572 9503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:45:17.807960 9678 tablet_service.cc:1467] Processing CreateTablet for tablet 42339b99279541cf938d32c3731927f1 (DEFAULT_TABLE table=test-table [id=bd56583716344a569ac19e1c6d392d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:17.808293 9678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42339b99279541cf938d32c3731927f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:17.810225 9621 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Bootstrap starting.
I20260808 00:45:17.810664 9621 tablet_bootstrap.cc:654] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:17.811874 9621 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: No bootstrap required, opened a new log
I20260808 00:45:17.811950 9621 ts_tablet_manager.cc:1397] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:45:17.812173 9621 raft_consensus.cc:348] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.812274 9621 raft_consensus.cc:374] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:17.812350 9621 raft_consensus.cc:729] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95afc8cb08544ef6bc208b962ac0a567, State: Initialized, Role: FOLLOWER
I20260808 00:45:17.812431 9621 consensus_queue.cc:260] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.812546 9621 ts_tablet_manager.cc:1428] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:17.812570 10194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.807596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46374 (local address 127.20.105.67:36287)
0808 00:45:17.807842 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.807846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.807856 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:45:17.807899 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.807901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.807917 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.808004 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:17.808366 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.809249 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.810774 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.811837 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.811931 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.812042 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.812363 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.812433 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":110,"threads_started":1}
I20260808 00:45:17.812774 10191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.807412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:36287 (local address 127.0.0.1:46374)
0808 00:45:17.807680 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.807691 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:17.807750 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:17.808236 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:17.808238 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:17.808240 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:17.808298 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.808299 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.809382 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.809383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.810658 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.810662 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.811982 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.811983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.812168 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.812196 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 00:45:17.812285 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:45:17.812476 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":94,"threads_started":1}
I20260808 00:45:17.813346 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.807247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39321 (local address 127.0.0.1:36818)
0808 00:45:17.807333 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.807349 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:17.807409 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:17.807896 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:17.807897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:17.807899 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:17.807972 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.807974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.809618 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.809618 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.810662 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.810664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.813062 (+ 2398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.813063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.813138 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.813143 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:45:17.813208 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 00:45:17.813277 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":1}
I20260808 00:45:17.813377 10192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.807431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36818 (local address 127.20.105.66:39321)
0808 00:45:17.807668 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.807675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.807688 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:45:17.807745 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.807748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.807769 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.807894 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:17.808156 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.809495 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.812104 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.812958 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.813083 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.813132 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.813231 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.813281 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":115,"threads_started":1}
I20260808 00:45:17.813891 10078 tablet_service.cc:1467] Processing CreateTablet for tablet 42339b99279541cf938d32c3731927f1 (DEFAULT_TABLE table=test-table [id=bd56583716344a569ac19e1c6d392d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:17.814260 10078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42339b99279541cf938d32c3731927f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:17.817040 9874 tablet_service.cc:1467] Processing CreateTablet for tablet 42339b99279541cf938d32c3731927f1 (DEFAULT_TABLE table=test-table [id=bd56583716344a569ac19e1c6d392d62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:17.817499 9874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42339b99279541cf938d32c3731927f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:17.818085 10196 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: Bootstrap starting.
I20260808 00:45:17.818794 10196 tablet_bootstrap.cc:654] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:17.819363 10196 log.cc:824] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:17.820444 10196 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: No bootstrap required, opened a new log
I20260808 00:45:17.820554 10196 ts_tablet_manager.cc:1397] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:45:17.821312 10199 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: Bootstrap starting.
I20260808 00:45:17.822119 10199 tablet_bootstrap.cc:654] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:17.822820 10199 log.cc:824] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:17.822747 10196 raft_consensus.cc:348] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.822950 10196 raft_consensus.cc:374] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:17.823011 10196 raft_consensus.cc:729] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048133867c124634bf30e6520e8d282f, State: Initialized, Role: FOLLOWER
I20260808 00:45:17.823166 10196 consensus_queue.cc:260] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.823504 10196 ts_tablet_manager.cc:1428] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:45:17.823573 10179 heartbeater.cc:503] Master 127.20.105.126:40993 was elected leader, sending a full tablet report...
I20260808 00:45:17.823956 10199 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: No bootstrap required, opened a new log
I20260808 00:45:17.824074 10199 ts_tablet_manager.cc:1397] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:45:17.825742 10199 raft_consensus.cc:348] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.825887 10199 raft_consensus.cc:374] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:17.825911 10199 raft_consensus.cc:729] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90f0fdb865484d02bfa134d61875c14a, State: Initialized, Role: FOLLOWER
I20260808 00:45:17.826066 10199 consensus_queue.cc:260] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.826396 10199 ts_tablet_manager.cc:1428] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:45:17.826522 9983 heartbeater.cc:503] Master 127.20.105.126:40993 was elected leader, sending a full tablet report...
I20260808 00:45:17.828431 10201 raft_consensus.cc:482] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:17.828578 10201 raft_consensus.cc:504] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.828986 10201 leader_election.cc:288] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321), 95afc8cb08544ef6bc208b962ac0a567 (127.20.105.65:35589)
I20260808 00:45:17.833097 10192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.829281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:39855 (local address 127.20.105.66:39321)
0808 00:45:17.829338 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.829341 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.829352 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:45:17.829386 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.829387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.829401 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.829503 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:17.829682 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.830581 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.832081 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.832681 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.832763 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.832778 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.832965 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.833019 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:45:17.833153 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.829385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:40339 (local address 127.20.105.65:35589)
0808 00:45:17.829454 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.829457 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.829549 (+ 92us) server_negotiation.cc:407] Connection header received
0808 00:45:17.829603 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.829605 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.829632 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.829737 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:17.829890 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.830742 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.832086 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.832799 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.832869 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.832985 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.833008 (+ 23us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.833067 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:45:17.833139 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.829172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39321 (local address 127.20.105.67:39855)
0808 00:45:17.829216 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.829231 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:17.829279 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:17.829509 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:17.829510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:17.829512 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:17.829608 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.829610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.830671 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.830672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.831928 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.831931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.832777 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.832778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.832878 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.832884 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:45:17.832940 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:45:17.833007 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":5}
I20260808 00:45:17.833148 10208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.829284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:35589 (local address 127.20.105.67:40339)
0808 00:45:17.829465 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.829475 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:17.829541 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:17.829752 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:17.829752 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:17.829754 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:17.829813 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.829815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.830813 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.830814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.831928 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:17.831931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.832882 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.832883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:17.832959 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.832962 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:45:17.833003 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 00:45:17.833036 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":116,"threads_started":1}
I20260808 00:45:17.833559 9904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42339b99279541cf938d32c3731927f1" candidate_uuid: "048133867c124634bf30e6520e8d282f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f0fdb865484d02bfa134d61875c14a" is_pre_election: true
I20260808 00:45:17.833532 9707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42339b99279541cf938d32c3731927f1" candidate_uuid: "048133867c124634bf30e6520e8d282f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95afc8cb08544ef6bc208b962ac0a567" is_pre_election: true
I20260808 00:45:17.833748 9904 raft_consensus.cc:2454] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048133867c124634bf30e6520e8d282f in term 0.
I20260808 00:45:17.833748 9707 raft_consensus.cc:2454] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048133867c124634bf30e6520e8d282f in term 0.
I20260808 00:45:17.833997 10011 leader_election.cc:302] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048133867c124634bf30e6520e8d282f, 90f0fdb865484d02bfa134d61875c14a; no voters:
I20260808 00:45:17.834172 10201 raft_consensus.cc:2781] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:45:17.834239 10201 raft_consensus.cc:482] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:45:17.834267 10201 raft_consensus.cc:3037] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:17.835201 10201 raft_consensus.cc:504] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.835372 10201 leader_election.cc:288] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [CANDIDATE]: Term 1 election: Requested vote from peers 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321), 95afc8cb08544ef6bc208b962ac0a567 (127.20.105.65:35589)
I20260808 00:45:17.835562 9904 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42339b99279541cf938d32c3731927f1" candidate_uuid: "048133867c124634bf30e6520e8d282f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90f0fdb865484d02bfa134d61875c14a"
I20260808 00:45:17.835615 9707 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42339b99279541cf938d32c3731927f1" candidate_uuid: "048133867c124634bf30e6520e8d282f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95afc8cb08544ef6bc208b962ac0a567"
I20260808 00:45:17.835700 9707 raft_consensus.cc:3037] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:17.835678 9904 raft_consensus.cc:3037] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:17.836616 9707 raft_consensus.cc:2454] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048133867c124634bf30e6520e8d282f in term 1.
I20260808 00:45:17.836699 9904 raft_consensus.cc:2454] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048133867c124634bf30e6520e8d282f in term 1.
I20260808 00:45:17.836829 10011 leader_election.cc:302] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048133867c124634bf30e6520e8d282f, 95afc8cb08544ef6bc208b962ac0a567; no voters:
I20260808 00:45:17.836937 10201 raft_consensus.cc:2781] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:17.837150 10201 raft_consensus.cc:686] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 1 LEADER]: Becoming Leader. State: Replica: 048133867c124634bf30e6520e8d282f, State: Running, Role: LEADER
I20260808 00:45:17.837284 10201 consensus_queue.cc:237] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:17.838394 9499 catalog_manager.cc:5495] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f reported cstate change: term changed from 0 to 1, leader changed from <none> to 048133867c124634bf30e6520e8d282f (127.20.105.67). New cstate: current_term: 1 leader_uuid: "048133867c124634bf30e6520e8d282f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } health_report { overall_health: UNKNOWN } } }
I20260808 00:45:17.845758 10194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.842921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46378 (local address 127.20.105.67:36287)
0808 00:45:17.842995 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:17.842997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:17.843007 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:45:17.843111 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:17.843112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:17.843133 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:17.843224 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:45:17.843388 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.843985 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.844817 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:17.845042 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:17.845133 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:17.845319 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:45:17.845463 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:45:17.845571 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:17.845589 (+ 18us) server_negotiation.cc:299] Negotiation successful
0808 00:45:17.845615 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:45:17.848448 9904 raft_consensus.cc:1261] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 1 FOLLOWER]: Refusing update from remote peer 048133867c124634bf30e6520e8d282f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:17.848448 9707 raft_consensus.cc:1261] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Refusing update from remote peer 048133867c124634bf30e6520e8d282f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:17.848891 10201 consensus_queue.cc:1035] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [LEADER]: Connected to new peer: Peer: permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:17.849089 10209 consensus_queue.cc:1035] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [LEADER]: Connected to new peer: Peer: permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:17.852341 10213 mvcc.cc:205] Tried to move back new op lower bound from 7316070063502741504 to 7316070063462625280. Current Snapshot: MvccSnapshot[applied={T|T < 7316070063502741504}]
I20260808 00:45:17.855479 9503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:17.855623 9503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:17.856598 9503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Sending DeleteTablet for 3 replicas of tablet 42339b99279541cf938d32c3731927f1
I20260808 00:45:17.857066 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 9556
I20260808 00:45:17.857048 10078 tablet_service.cc:1514] Processing DeleteTablet for tablet 42339b99279541cf938d32c3731927f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:46374
I20260808 00:45:17.857714 9874 tablet_service.cc:1514] Processing DeleteTablet for tablet 42339b99279541cf938d32c3731927f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:17 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:36818
I20260808 00:45:17.869776 10226 tablet_replica.cc:330] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: stopping tablet replica
I20260808 00:45:17.870113 10226 raft_consensus.cc:2229] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:45:17.870405 10226 raft_consensus.cc:2258] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:17.872192 10226 ts_tablet_manager.cc:1905] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:17.872426 10228 tablet_replica.cc:330] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: stopping tablet replica
I20260808 00:45:17.872653 10228 raft_consensus.cc:2229] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:17.872838 10228 raft_consensus.cc:2258] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:17.874428 10228 ts_tablet_manager.cc:1905] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:17.876099 10226 ts_tablet_manager.cc:1918] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:17.876260 10226 log.cc:1196] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/42339b99279541cf938d32c3731927f1
I20260808 00:45:17.876641 10228 ts_tablet_manager.cc:1918] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:17.876737 10228 log.cc:1196] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/42339b99279541cf938d32c3731927f1
I20260808 00:45:17.876907 10226 ts_tablet_manager.cc:1939] T 42339b99279541cf938d32c3731927f1 P 048133867c124634bf30e6520e8d282f: Deleting consensus metadata
I20260808 00:45:17.877077 10228 ts_tablet_manager.cc:1939] T 42339b99279541cf938d32c3731927f1 P 90f0fdb865484d02bfa134d61875c14a: Deleting consensus metadata
I20260808 00:45:17.877631 9484 catalog_manager.cc:4841] TS 048133867c124634bf30e6520e8d282f (127.20.105.67:36287): tablet 42339b99279541cf938d32c3731927f1 (table test-table [id=bd56583716344a569ac19e1c6d392d62]) successfully deleted
I20260808 00:45:17.877755 9484 catalog_manager.cc:4841] TS 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321): tablet 42339b99279541cf938d32c3731927f1 (table test-table [id=bd56583716344a569ac19e1c6d392d62]) successfully deleted
W20260808 00:45:17.881776 9484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:35589 (error 108)
I20260808 00:45:17.882009 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 9778
I20260808 00:45:17.883109 10191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:17.882912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:35589 (local address 127.0.0.1:37324)
0808 00:45:17.882980 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:17.883026 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:35589: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 00:45:17.883245 9484 catalog_manager.cc:4584] TS 95afc8cb08544ef6bc208b962ac0a567 (127.20.105.65:35589): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 42339b99279541cf938d32c3731927f1: Network error: Client connection negotiation failed: client connection to 127.20.105.65:35589: connect: Connection refused (error 111)
I20260808 00:45:17.894007 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 9986
I20260808 00:45:17.927273 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 9441
I20260808 00:45:17.941967 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40993
--webserver_interface=127.20.105.126
--webserver_port=39385
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:40993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:18.048931 10230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:18.049324 10230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:18.049374 10230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:18.051210 10230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:18.051331 10230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:18.051364 10230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:18.051399 10230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:18.051434 10230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:18.053646 10230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:40993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=39385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:18.054075 10230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:18.054415 10230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:18.060984 10238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:18.060994 10240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:18.060995 10236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:18.061551 10230 server_base.cc:1034] running on GCE node
I20260808 00:45:18.061934 10230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:18.062399 10230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:18.063632 10230 hybrid_clock.cc:648] HybridClock initialized: now 1786149918063619 us; error 65 us; skew 500 ppm
Aug 08 00:45:18 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149918, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:18.067029 10230 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:18.067648 10230 webserver.cc:460] Webserver started at http://127.20.105.126:39385/ using document root <none> and password file <none>
I20260808 00:45:18.067946 10230 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:18.068005 10230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:18.069895 10230 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:18.070837 10250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:18.071098 10230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:18.071226 10230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "bf6858f8137c4ac5b218b6eeeade2ed4"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d7099846f6a6ae0f03a7dc22abb00999"
server_key_iv: "1984cced6cc52569bf409cc4ff176fdf"
server_key_version: "encryptionkey@0"
I20260808 00:45:18.071751 10230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:18.090046 10230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:18.093083 10230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:18.093307 10230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:18.098914 10230 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:40993
I20260808 00:45:18.098956 10318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:40993 every 8 connection(s)
I20260808 00:45:18.099501 10230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:18.100703 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 10230
I20260808 00:45:18.101107 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:35589
--local_ip_for_outbound_sockets=127.20.105.65
--tserver_master_addrs=127.20.105.126:40993
--webserver_port=36103
--webserver_interface=127.20.105.65
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:18.101925 10319 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:45:18.102787 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Bootstrap starting.
I20260808 00:45:18.116469 10319 log.cc:824] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:18.120986 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:45:18.121511 10319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4: Bootstrap complete.
I20260808 00:45:18.125540 10319 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:18.126045 10319 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6858f8137c4ac5b218b6eeeade2ed4, State: Initialized, Role: FOLLOWER
I20260808 00:45:18.126317 10319 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:18.126435 10319 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:45:18.126487 10319 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:45:18.126543 10319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:45:18.128504 10319 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:18.128654 10319 leader_election.cc:302] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf6858f8137c4ac5b218b6eeeade2ed4; no voters:
I20260808 00:45:18.128887 10319 leader_election.cc:288] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 00:45:18.128983 10324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:45:18.129181 10324 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [term 3 LEADER]: Becoming Leader. State: Replica: bf6858f8137c4ac5b218b6eeeade2ed4, State: Running, Role: LEADER
I20260808 00:45:18.129226 10319 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:18.129321 10324 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } }
I20260808 00:45:18.129753 10325 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } } }
I20260808 00:45:18.129844 10325 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:18.130139 10326 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf6858f8137c4ac5b218b6eeeade2ed4. Latest consensus state: current_term: 3 leader_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6858f8137c4ac5b218b6eeeade2ed4" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40993 } } }
I20260808 00:45:18.130295 10326 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf6858f8137c4ac5b218b6eeeade2ed4 [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:18.130383 10338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:18.131495 10338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:45:18.131645 10338 catalog_manager.cc:1259] Loaded cluster ID: 28e1cd8651ba41568fceee82f69a7d0a
I20260808 00:45:18.131685 10338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 00:45:18.132187 10338 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff9138509ad google::LogMessage::Fail() at ??:0
@ 0x7ff91385388a google::LogMessage::SendToLog() at ??:0
@ 0x7ff9138504de google::LogMessage::Flush() at ??:0
@ 0x7ff913851a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff91b7710f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7ff91b77a8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff91571e0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff915713954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff9165776db start_thread at ??:0
@ 0x7ff91234e61f clone at ??:0
W20260808 00:45:18.251739 10321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:18.252175 10321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:18.252226 10321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:18.254438 10321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:18.254520 10321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:18.254575 10321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:45:18.257476 10321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:35589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=36103
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:18.257930 10321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:18.258347 10321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:18.265916 10372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:18.266227 10321 server_base.cc:1034] running on GCE node
W20260808 00:45:18.265941 10373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:18.265926 10376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:18.266726 10321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:18.267072 10321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:18.268261 10321 hybrid_clock.cc:648] HybridClock initialized: now 1786149918268229 us; error 52 us; skew 500 ppm
Aug 08 00:45:18 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149918, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:18.271806 10321 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:45:18.272496 10321 webserver.cc:460] Webserver started at http://127.20.105.65:36103/ using document root <none> and password file <none>
I20260808 00:45:18.272779 10321 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:18.272859 10321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:18.274657 10321 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:45:18.275614 10383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:18.275832 10321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:18.275956 10321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "95afc8cb08544ef6bc208b962ac0a567"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "23af1199b1210afb58861b5c2b13a066"
server_key_iv: "6ddd19439b3517ce7baf62ddd0f5bf21"
server_key_version: "encryptionkey@0"
I20260808 00:45:18.276535 10321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:18.292335 10321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:18.295285 10321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:18.295491 10321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:18.295949 10321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:18.296638 10397 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:45:18.299168 10321 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:45:18.299263 10321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 00:45:18.299320 10321 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:45:18.300365 10321 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:45:18.300436 10321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:18.300536 10397 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Bootstrap starting.
I20260808 00:45:18.310518 10321 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:35589
I20260808 00:45:18.311113 10321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:45:18.311730 10570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:35589 every 8 connection(s)
I20260808 00:45:18.315443 10397 log.cc:824] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:18.315518 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 10321
I20260808 00:45:18.315924 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:39321
--local_ip_for_outbound_sockets=127.20.105.66
--tserver_master_addrs=127.20.105.126:40993
--webserver_port=35589
--webserver_interface=127.20.105.66
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:18.319161 10397 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:45:18.319551 10397 tablet_bootstrap.cc:492] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Bootstrap complete.
I20260808 00:45:18.320926 10397 ts_tablet_manager.cc:1397] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.016s
I20260808 00:45:18.322743 10397 raft_consensus.cc:348] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:18.323176 10397 raft_consensus.cc:729] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95afc8cb08544ef6bc208b962ac0a567, State: Initialized, Role: FOLLOWER
I20260808 00:45:18.323405 10397 consensus_queue.cc:260] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:18.323763 10397 ts_tablet_manager.cc:1428] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260808 00:45:18.425710 10576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:18.426077 10576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:18.426134 10576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:18.427809 10576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:18.427924 10576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:18.427985 10576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:45:18.430168 10576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:39321
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=35589
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:18.430591 10576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:18.430951 10576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:18.437918 10587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:18.437958 10586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:18.438386 10576 server_base.cc:1034] running on GCE node
W20260808 00:45:18.437980 10589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:18.438750 10576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:18.439157 10576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:18.440371 10576 hybrid_clock.cc:648] HybridClock initialized: now 1786149918440316 us; error 69 us; skew 500 ppm
Aug 08 00:45:18 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149918, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:18.445000 10576 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:45:18.447912 10576 webserver.cc:460] Webserver started at http://127.20.105.66:35589/ using document root <none> and password file <none>
I20260808 00:45:18.448365 10576 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:18.448436 10576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:18.450409 10576 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:45:18.451239 10598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:18.451471 10576 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:18.451589 10576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "90f0fdb865484d02bfa134d61875c14a"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "5e22826aebd4c5829a795cb059fd35ef"
server_key_iv: "7138f1504c6a68702606313503f32bae"
server_key_version: "encryptionkey@0"
I20260808 00:45:18.452147 10576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260808 00:45:18.455130 10574 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:45:18.312230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:43301)
0808 00:45:18.312460 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.312520 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:18.313138 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:18.454834 (+141696us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":124,"threads_started":1}
W20260808 00:45:18.455541 10571 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:40993 (0 consecutive failures): Network error: Failed to ping master at 127.20.105.126:40993: Client connection negotiation failed: client connection to 127.20.105.126:40993: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:45:18.456079 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:18.455886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:53339)
0808 00:45:18.455962 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.456042 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 00:45:18.456595 10574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:18.456441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:41857)
0808 00:45:18.456513 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.456551 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 00:45:18.456779 10571 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:45:18.468396 10576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:18.471747 10576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:18.471909 10576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:18.472415 10576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:18.472913 10576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:18.472975 10576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:18.473030 10576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:18.473085 10576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:18.481093 10576 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:39321
I20260808 00:45:18.481138 10761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:39321 every 8 connection(s)
I20260808 00:45:18.481614 10576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:45:18.483047 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:18.482510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:38601)
0808 00:45:18.482734 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.482835 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":93,"threads_started":1}
W20260808 00:45:18.483584 10762 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:40993 (0 consecutive failures): Network error: Failed to ping master at 127.20.105.126:40993: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
I20260808 00:45:18.483922 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:18.483825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:39411)
0808 00:45:18.483874 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.483902 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:45:18.484336 10764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:18.484275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:38663)
0808 00:45:18.484298 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.484320 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260808 00:45:18.484465 10762 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:45:18.488593 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 10576
I20260808 00:45:18.488926 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:36287
--local_ip_for_outbound_sockets=127.20.105.67
--tserver_master_addrs=127.20.105.126:40993
--webserver_port=40603
--webserver_interface=127.20.105.67
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:18.593609 10765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:18.593896 10765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:18.593925 10765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:18.595160 10765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:18.595219 10765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:18.595250 10765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:45:18.596978 10765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:36287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=40603
--tserver_master_addrs=127.20.105.126:40993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:18.597307 10765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:18.597625 10765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:18.604027 10770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:18.604104 10771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:18.604104 10773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:18.604295 10765 server_base.cc:1034] running on GCE node
I20260808 00:45:18.604568 10765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:18.604862 10765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:18.606040 10765 hybrid_clock.cc:648] HybridClock initialized: now 1786149918606002 us; error 59 us; skew 500 ppm
Aug 08 00:45:18 dist-test-slave-kvfs krb5kdc[8639](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149918, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:18.609201 10765 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:45:18.609766 10765 webserver.cc:460] Webserver started at http://127.20.105.67:40603/ using document root <none> and password file <none>
I20260808 00:45:18.610047 10765 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:18.610124 10765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:18.611722 10765 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:45:18.612565 10783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:18.612879 10765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:45:18.613016 10765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "048133867c124634bf30e6520e8d282f"
format_stamp: "Formatted at 2026-08-08 00:45:16 on dist-test-slave-kvfs"
server_key: "d778a3f44dbdcb7bc70c378fa5f936e9"
server_key_iv: "8e637c1023a6d305f2cc2ef8782f3d61"
server_key_version: "encryptionkey@0"
I20260808 00:45:18.613495 10765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:18.623486 10765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:18.626196 10765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:18.626389 10765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:18.626703 10765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:18.627099 10765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:18.627158 10765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:18.627207 10765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:18.627238 10765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:18.635633 10765 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:36287
I20260808 00:45:18.635676 10955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:36287 every 8 connection(s)
I20260808 00:45:18.636221 10765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:45:18.637635 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 10765
I20260808 00:45:18.637679 10958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:18.637186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:56603)
0808 00:45:18.637380 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.637478 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":94,"threads_started":1}
W20260808 00:45:18.638173 10956 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:40993 (0 consecutive failures): Network error: Failed to ping master at 127.20.105.126:40993: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
I20260808 00:45:18.638504 10958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:18.638402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:43789)
0808 00:45:18.638454 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.638480 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 00:45:18.638810 10958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:18.638720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:41009)
0808 00:45:18.638766 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:18.638789 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:45:18.638928 10956 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:45:19.312526 10562 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.20.105.65:35589
I20260808 00:45:19.457691 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:19.457314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:44241)
0808 00:45:19.457510 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:19.457580 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":117,"threads_started":1}
I20260808 00:45:19.485304 10965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:19.484971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:35139)
0808 00:45:19.485154 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:19.485209 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":116,"threads_started":1}
I20260808 00:45:19.639725 10967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:19.639377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:56075)
0808 00:45:19.639587 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:19.639642 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":106,"threads_started":1}
I20260808 00:45:19.862757 10969 raft_consensus.cc:482] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:19.862943 10969 raft_consensus.cc:504] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:19.863380 10969 leader_election.cc:288] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321), 048133867c124634bf30e6520e8d282f (127.20.105.67:36287)
Aug 08 00:45:19 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149918, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.67@KRBTEST.COM
Aug 08 00:45:19 dist-test-slave-kvfs krb5kdc[8639](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149918, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.66@KRBTEST.COM
I20260808 00:45:19.876883 10973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:19.863985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:42101 (local address 127.20.105.67:36287)
0808 00:45:19.864190 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:19.864203 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:19.864241 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:45:19.864809 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:19.864812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:19.864861 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:19.865049 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:19.865953 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:19.867279 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:19.868572 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:19.869066 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:19.869221 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:19.872273 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:19.872279 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:19.872279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:19.872293 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:19.874389 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:19.875037 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:19.875042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:19.875042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:19.875199 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:19.875588 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:19.875589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:19.875590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:19.875962 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:19.876103 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:19.876577 (+ 474us) server_negotiation.cc:299] Negotiation successful
0808 00:45:19.876732 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":121,"threads_started":1}
I20260808 00:45:19.877151 10970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:19.863787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:50637 (local address 127.20.105.66:39321)
0808 00:45:19.864067 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:19.864085 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:19.864136 (+ 51us) server_negotiation.cc:407] Connection header received
0808 00:45:19.864758 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:19.864760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:19.864818 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:19.864962 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:19.865966 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:19.867312 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:19.868600 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:19.869067 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:19.869199 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:19.872406 (+ 3207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:19.872411 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:19.872411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:19.872423 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:19.874597 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:19.875069 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:19.875073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:19.875073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:19.875199 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:19.875569 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:19.875570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:19.875571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:19.875962 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:19.876102 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:19.876571 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 00:45:19.876733 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":122,"threads_started":1}
I20260808 00:45:19.877256 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:19.863880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:36287 (local address 127.20.105.65:42101)
0808 00:45:19.864059 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:19.864071 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:19.864148 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:19.865070 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:19.865072 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:19.865554 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:19.865801 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:19.865809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:19.867395 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:19.867397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:19.868478 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:19.868481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:19.869181 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:19.869183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:19.869314 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:19.870058 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:19.870081 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:19.872121 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:19.874508 (+ 2387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:19.874518 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:19.874521 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:19.874906 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:19.875281 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:19.875281 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:19.875282 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:19.875468 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:19.876109 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:19.876122 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:19.876334 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 00:45:19.876483 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 00:45:19.876851 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":243,"thread_start_us":92,"threads_started":1}
I20260808 00:45:19.877766 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:19.863658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39321 (local address 127.20.105.65:50637)
0808 00:45:19.863731 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:19.863747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:19.863831 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:19.865065 (+ 1234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:19.865069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:19.865546 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:19.865801 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:19.865809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:19.867428 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:19.867432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:19.868487 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:19.868490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:19.869176 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:19.869178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:19.869312 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:19.870118 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:19.870124 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:19.872279 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:19.874747 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:19.874749 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:19.874749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:19.874978 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:19.875278 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:19.875278 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:19.875279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:19.875468 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:19.876109 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:19.876122 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:19.876334 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 00:45:19.876484 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 00:45:19.876756 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":53}
W20260808 00:45:19.878625 10389 leader_election.cc:341] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 048133867c124634bf30e6520e8d282f (127.20.105.67:36287): Not found: Tablet not found: 42339b99279541cf938d32c3731927f1
W20260808 00:45:19.878726 10389 leader_election.cc:341] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321): Not found: Tablet not found: 42339b99279541cf938d32c3731927f1
I20260808 00:45:19.878780 10389 leader_election.cc:302] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95afc8cb08544ef6bc208b962ac0a567; no voters: 048133867c124634bf30e6520e8d282f, 90f0fdb865484d02bfa134d61875c14a
I20260808 00:45:19.878954 10969 raft_consensus.cc:2726] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:45:20.458684 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:20.458357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:51437)
0808 00:45:20.458537 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:20.458604 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260808 00:45:20.486231 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:20.485932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:33175)
0808 00:45:20.486114 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:20.486167 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":84,"threads_started":1}
I20260808 00:45:20.640758 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:20.640424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:34839)
0808 00:45:20.640635 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:20.640691 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":103,"threads_started":1}
I20260808 00:45:21.459746 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:21.459432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:39433)
0808 00:45:21.459622 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:21.459674 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":104,"threads_started":1}
I20260808 00:45:21.487100 10981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:21.486818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:55907)
0808 00:45:21.486995 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:21.487036 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":112,"threads_started":1}
I20260808 00:45:21.641729 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:21.641419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:47337)
0808 00:45:21.641599 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:21.641654 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260808 00:45:22.054216 10983 raft_consensus.cc:482] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:22.054412 10983 raft_consensus.cc:504] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:22.054680 10983 leader_election.cc:288] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321), 048133867c124634bf30e6520e8d282f (127.20.105.67:36287)
W20260808 00:45:22.055379 10389 leader_election.cc:341] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321): Not found: Tablet not found: 42339b99279541cf938d32c3731927f1
W20260808 00:45:22.055483 10389 leader_election.cc:341] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 048133867c124634bf30e6520e8d282f (127.20.105.67:36287): Not found: Tablet not found: 42339b99279541cf938d32c3731927f1
I20260808 00:45:22.055501 10389 leader_election.cc:302] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95afc8cb08544ef6bc208b962ac0a567; no voters: 048133867c124634bf30e6520e8d282f, 90f0fdb865484d02bfa134d61875c14a
I20260808 00:45:22.055579 10983 raft_consensus.cc:2726] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:45:22.460865 10984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:22.460473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:40127)
0808 00:45:22.460725 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:22.460789 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":132,"threads_started":1}
I20260808 00:45:22.487929 10985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:22.487637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:58185)
0808 00:45:22.487804 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:22.487841 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260808 00:45:22.642764 10986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:22.642388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:55955)
0808 00:45:22.642623 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:22.642688 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260808 00:45:23.461921 10989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:23.461561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:43849)
0808 00:45:23.461796 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:23.461849 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":122,"threads_started":1}
I20260808 00:45:23.488826 10990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:23.488525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:35305)
0808 00:45:23.488714 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:23.488756 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":134,"threads_started":1}
I20260808 00:45:23.643787 10991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:23.643447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:43377)
0808 00:45:23.643659 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:23.643713 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":112,"threads_started":1}
I20260808 00:45:24.462961 10992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:24.462627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:32783)
0808 00:45:24.462828 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:24.462879 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":101,"threads_started":1}
I20260808 00:45:24.489857 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:24.489463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:52551)
0808 00:45:24.489717 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:24.489777 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":141,"threads_started":1}
I20260808 00:45:24.644810 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:24.644484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:43967)
0808 00:45:24.644686 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:24.644737 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
I20260808 00:45:25.166383 10995 raft_consensus.cc:482] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:25.166548 10995 raft_consensus.cc:504] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:25.166878 10995 leader_election.cc:288] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321), 048133867c124634bf30e6520e8d282f (127.20.105.67:36287)
W20260808 00:45:25.167485 10389 leader_election.cc:341] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 048133867c124634bf30e6520e8d282f (127.20.105.67:36287): Not found: Tablet not found: 42339b99279541cf938d32c3731927f1
W20260808 00:45:25.167599 10389 leader_election.cc:341] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321): Not found: Tablet not found: 42339b99279541cf938d32c3731927f1
I20260808 00:45:25.167642 10389 leader_election.cc:302] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95afc8cb08544ef6bc208b962ac0a567; no voters: 048133867c124634bf30e6520e8d282f, 90f0fdb865484d02bfa134d61875c14a
I20260808 00:45:25.167742 10995 raft_consensus.cc:2726] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:45:25.464175 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:25.463758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:54281)
0808 00:45:25.463980 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:25.464099 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":122,"threads_started":1}
I20260808 00:45:25.490826 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:25.490500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:44311)
0808 00:45:25.490721 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:25.490762 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
I20260808 00:45:25.645941 11001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:25.645584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:47767)
0808 00:45:25.645818 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:25.645871 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":109,"threads_started":1}
I20260808 00:45:26.465158 11002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:26.464842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:42127)
0808 00:45:26.465025 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:26.465081 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260808 00:45:26.491729 11003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:26.491399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:42119)
0808 00:45:26.491603 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:26.491655 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":89,"threads_started":1}
I20260808 00:45:26.646920 11005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:26.646614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:48015)
0808 00:45:26.646802 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:26.646851 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":82,"threads_started":1}
I20260808 00:45:27.466104 11007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:27.465774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:36201)
0808 00:45:27.465981 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:27.466029 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":94,"threads_started":1}
I20260808 00:45:27.492766 11008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:27.492408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:53557)
0808 00:45:27.492633 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:27.492680 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":144,"threads_started":1}
I20260808 00:45:27.648133 11009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:27.647713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:33799)
0808 00:45:27.647948 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:27.647991 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":84,"threads_started":1}
I20260808 00:45:28.467059 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:28.466783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:54221)
0808 00:45:28.466948 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:28.466985 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":101,"threads_started":1}
I20260808 00:45:28.493714 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:28.493420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:50403)
0808 00:45:28.493595 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:28.493643 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":102,"threads_started":1}
I20260808 00:45:28.649068 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:28.648723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:52759)
0808 00:45:28.648907 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:28.648955 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":91,"threads_started":1}
I20260808 00:45:29.468155 11015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:29.467732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:47401)
0808 00:45:29.467946 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:29.468008 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":104,"threads_started":1}
I20260808 00:45:29.494575 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:29.494251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:54895)
0808 00:45:29.494437 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:29.494490 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260808 00:45:29.650112 11017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:29.649775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:44163)
0808 00:45:29.649979 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:29.650037 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
I20260808 00:45:29.887182 11018 raft_consensus.cc:482] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:29.887349 11018 raft_consensus.cc:504] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90f0fdb865484d02bfa134d61875c14a" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39321 } } peers { permanent_uuid: "048133867c124634bf30e6520e8d282f" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 36287 } } peers { permanent_uuid: "95afc8cb08544ef6bc208b962ac0a567" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35589 } }
I20260808 00:45:29.887640 11018 leader_election.cc:288] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321), 048133867c124634bf30e6520e8d282f (127.20.105.67:36287)
W20260808 00:45:29.888262 10389 leader_election.cc:341] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 048133867c124634bf30e6520e8d282f (127.20.105.67:36287): Not found: Tablet not found: 42339b99279541cf938d32c3731927f1
W20260808 00:45:29.888334 10389 leader_election.cc:341] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 90f0fdb865484d02bfa134d61875c14a (127.20.105.66:39321): Not found: Tablet not found: 42339b99279541cf938d32c3731927f1
I20260808 00:45:29.888352 10389 leader_election.cc:302] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 95afc8cb08544ef6bc208b962ac0a567; no voters: 048133867c124634bf30e6520e8d282f, 90f0fdb865484d02bfa134d61875c14a
I20260808 00:45:29.888435 11018 raft_consensus.cc:2726] T 42339b99279541cf938d32c3731927f1 P 95afc8cb08544ef6bc208b962ac0a567 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:45:30.469163 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:30.468799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:33573)
0808 00:45:30.469019 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:30.469077 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":105,"threads_started":1}
I20260808 00:45:30.495537 11020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:30.495219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:36625)
0808 00:45:30.495417 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:30.495463 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":129,"threads_started":1}
I20260808 00:45:30.651072 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:30.650783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:42047)
0808 00:45:30.650954 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:30.651002 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":81,"threads_started":1}
I20260808 00:45:31.470305 11023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:31.469976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:45909)
0808 00:45:31.470175 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:31.470233 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260808 00:45:31.496649 11024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:31.496354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:35595)
0808 00:45:31.496521 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:31.496572 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260808 00:45:31.651965 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:31.651666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:50011)
0808 00:45:31.651836 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:31.651889 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":120,"threads_started":1}
I20260808 00:45:32.471323 11028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:32.471005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:54097)
0808 00:45:32.471199 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:32.471251 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":100,"threads_started":1}
I20260808 00:45:32.497661 11029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:32.497323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:51985)
0808 00:45:32.497523 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:32.497586 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260808 00:45:32.652827 11030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:32.652536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:37247)
0808 00:45:32.652705 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:32.652754 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":88,"threads_started":1}
I20260808 00:45:33.472646 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:33.472344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.65:50095)
0808 00:45:33.472520 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:33.472572 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":85,"threads_started":1}
I20260808 00:45:33.498795 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:33.498466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.66:34163)
0808 00:45:33.498668 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:33.498723 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260808 00:45:33.648921 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 10321
I20260808 00:45:33.655287 11033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:33.654950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40993 (local address 127.20.105.67:45163)
0808 00:45:33.655145 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:33.655196 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:40993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
I20260808 00:45:33.665544 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 10576
I20260808 00:45:33.682186 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 10765
2026-08-08T00:45:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19499 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:45:34 dist-test-slave-kvfs krb5kdc[11037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:45:34 dist-test-slave-kvfs krb5kdc[11037](info): set up 2 sockets
Aug 08 00:45:34 dist-test-slave-kvfs krb5kdc[11037](info): commencing operation
krb5kdc: starting...
W20260808 00:45:35.287063 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:45:35 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:45:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:45:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:45:35.444928 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34797
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:34797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:35.547708 11059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:35.548097 11059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:35.548146 11059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:35.549810 11059 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:45:35.549930 11059 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:45:35.549960 11059 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:45:35.549983 11059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:35.550009 11059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:35.550041 11059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:35.550072 11059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:35.550102 11059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:35.552587 11059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:34797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:35.553164 11059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:35.553494 11059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:35.559578 11068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:35.559587 11069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:35.559587 11071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:35.559623 11059 server_base.cc:1034] running on GCE node
I20260808 00:45:35.560227 11059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:35.560667 11059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:35.561897 11059 hybrid_clock.cc:648] HybridClock initialized: now 1786149935561830 us; error 58 us; skew 500 ppm
Aug 08 00:45:35 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149935, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:35.569514 11059 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:35.570261 11059 webserver.cc:460] Webserver started at http://127.20.105.126:42823/ using document root <none> and password file <none>
I20260808 00:45:35.570573 11059 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:35.570652 11059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:35.570824 11059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:35.572255 11059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "171543b6fa974aa0a9919280b50a9b9f"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "33087359d923cd7723077f15818d3cd6"
server_key_iv: "9b9e26ca745b3f4c592e13e8152f92fb"
server_key_version: "encryptionkey@0"
I20260808 00:45:35.572700 11059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "171543b6fa974aa0a9919280b50a9b9f"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "33087359d923cd7723077f15818d3cd6"
server_key_iv: "9b9e26ca745b3f4c592e13e8152f92fb"
server_key_version: "encryptionkey@0"
I20260808 00:45:35.574385 11059 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:45:35.575409 11082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:35.575636 11059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:35.575757 11059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "171543b6fa974aa0a9919280b50a9b9f"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "33087359d923cd7723077f15818d3cd6"
server_key_iv: "9b9e26ca745b3f4c592e13e8152f92fb"
server_key_version: "encryptionkey@0"
I20260808 00:45:35.575845 11059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:35.598232 11059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:35.601227 11059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:35.601459 11059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:35.606952 11059 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:34797
I20260808 00:45:35.607003 11157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:34797 every 8 connection(s)
I20260808 00:45:35.607488 11059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:35.608776 11158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:35.611799 11158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Bootstrap starting.
I20260808 00:45:35.612716 11158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:35.613107 11158 log.cc:824] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:35.614071 11158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: No bootstrap required, opened a new log
I20260808 00:45:35.614401 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 11059
I20260808 00:45:35.614526 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:45:35.614766 20901 external_mini_cluster.cc:1428] Setting key 19225973f309e75d092d553faba716fc
I20260808 00:45:35.616736 11158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:45:35.616945 11158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:35.617018 11158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 171543b6fa974aa0a9919280b50a9b9f, State: Initialized, Role: FOLLOWER
I20260808 00:45:35.617247 11158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:45:35.617348 11158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:45:35.617374 11158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:45:35.617427 11158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:35.618722 11158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:45:35.618886 11158 leader_election.cc:302] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 171543b6fa974aa0a9919280b50a9b9f; no voters:
I20260808 00:45:35.619108 11158 leader_election.cc:288] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:45:35.619179 11164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:35.619364 11164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 1 LEADER]: Becoming Leader. State: Replica: 171543b6fa974aa0a9919280b50a9b9f, State: Running, Role: LEADER
I20260808 00:45:35.619467 11158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:35.619557 11164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:45:35.620107 11164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 171543b6fa974aa0a9919280b50a9b9f. Latest consensus state: current_term: 1 leader_uuid: "171543b6fa974aa0a9919280b50a9b9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } } }
I20260808 00:45:35.620606 11164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:35.620525 11166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "171543b6fa974aa0a9919280b50a9b9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } } }
I20260808 00:45:35.621066 11166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: This master's current role is: LEADER
W20260808 00:45:35.621790 11192 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:45:35.621997 11192 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:45:35.622087 11181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:35.622327 11181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:45:35 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:35.624387 11181 catalog_manager.cc:1347] Generated new cluster ID: c38bf853828d4318b9a4f6ed8ef6bd68
I20260808 00:45:35.624444 11181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:45:35.629067 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:35.615472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47860 (local address 127.20.105.126:34797)
0808 00:45:35.615854 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:35.615865 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:35.615924 (+ 59us) server_negotiation.cc:407] Connection header received
0808 00:45:35.616726 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:35.616739 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:35.616824 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:35.617110 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:35.617994 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:35.619207 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:35.620451 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:35.621192 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:35.621770 (+ 578us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:35.624819 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:35.624824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:35.624824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:35.624838 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:35.626845 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:35.627234 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:35.627238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:35.627238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:35.627352 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:35.627561 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:35.627562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:35.627562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:35.627908 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:35.628088 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:35.628647 (+ 559us) server_negotiation.cc:299] Negotiation successful
0808 00:45:35.628801 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":256,"threads_started":1}
I20260808 00:45:35.652047 11181 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:45:35.652993 11181 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:45:35.668064 11181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Generated new TSK 0
I20260808 00:45:35.668421 11181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:45:35.712931 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34797
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:35.819412 11198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:35.819768 11198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:35.819818 11198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:35.821391 11198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:35.821475 11198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:35.821529 11198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:45:35.823321 11198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:35.823678 11198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:35.823988 11198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:35.830507 11208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:35.830508 11211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:35.830507 11206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:35.830607 11198 server_base.cc:1034] running on GCE node
I20260808 00:45:35.831027 11198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:35.831326 11198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:35.832593 11198 hybrid_clock.cc:648] HybridClock initialized: now 1786149935832521 us; error 45 us; skew 500 ppm
Aug 08 00:45:35 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149935, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:35.835996 11198 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:45:35.836621 11198 webserver.cc:460] Webserver started at http://127.20.105.65:42329/ using document root <none> and password file <none>
I20260808 00:45:35.836922 11198 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:35.836974 11198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:35.837123 11198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:35.838420 11198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "fa3de2302cb84028b2b60596fe36e229"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "e818fb19683185a8f0c45839a39719e0"
server_key_iv: "071331a0ba3aa3a2b7b8faeff5a142f6"
server_key_version: "encryptionkey@0"
I20260808 00:45:35.838843 11198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "fa3de2302cb84028b2b60596fe36e229"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "e818fb19683185a8f0c45839a39719e0"
server_key_iv: "071331a0ba3aa3a2b7b8faeff5a142f6"
server_key_version: "encryptionkey@0"
I20260808 00:45:35.840498 11198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:45:35.841440 11220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:35.841738 11198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:35.841845 11198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "fa3de2302cb84028b2b60596fe36e229"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "e818fb19683185a8f0c45839a39719e0"
server_key_iv: "071331a0ba3aa3a2b7b8faeff5a142f6"
server_key_version: "encryptionkey@0"
I20260808 00:45:35.841948 11198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:35.870208 11198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:35.873206 11198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:35.873373 11198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:35.873651 11198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:35.874097 11198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:35.874171 11198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:35.874228 11198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:35.874277 11198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:35.882292 11198 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:37417
I20260808 00:45:35.882375 11384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:37417 every 8 connection(s)
I20260808 00:45:35.882828 11198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:45:35.883324 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 11198
I20260808 00:45:35.883462 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:45:35.883718 20901 external_mini_cluster.cc:1428] Setting key c232d133421baf82daee721389bd33ca
Aug 08 00:45:35 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149935, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:35.896006 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:35.884270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:54933 (local address 127.20.105.126:34797)
0808 00:45:35.884348 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:35.884351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:35.884549 (+ 198us) server_negotiation.cc:407] Connection header received
0808 00:45:35.885235 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:35.885237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:35.885256 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:35.885324 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:35.886626 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:35.887273 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:35.888619 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:35.888891 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:35.888992 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:35.892404 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:35.892407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:35.892407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:35.892416 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:35.894326 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:35.894812 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:35.894814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:35.894814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:35.894887 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:35.895163 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:35.895165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:35.895165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:35.895350 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:35.895426 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:35.895753 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 00:45:35.895865 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:45:35.896229 11388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:35.883865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:54933)
0808 00:45:35.884395 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:35.884453 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:35.885117 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:35.885428 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:35.885433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:35.886229 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:35.886506 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:35.886513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:35.887395 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:35.887396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:35.888517 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:35.888520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:35.889015 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:35.889016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:35.889204 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:35.889943 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:35.889956 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:35.892252 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:35.894435 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:35.894445 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:35.894448 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:35.894713 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:35.894972 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:35.894973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:35.894974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:35.895090 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:35.895430 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:35.895436 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:35.895591 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 00:45:35.895680 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:45:35.895838 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":116,"threads_started":1}
I20260808 00:45:35.896941 11385 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34797
I20260808 00:45:35.897106 11385 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:35.897370 11385 heartbeater.cc:511] Master 127.20.105.126:34797 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
I20260808 00:45:35.898200 11105 ts_manager.cc:194] Registered new tserver with Master: fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417)
I20260808 00:45:35.899083 11105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:54933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:45:35.924005 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34797
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:36.028121 11392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:36.028390 11392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:36.028419 11392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:36.029752 11392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:36.029798 11392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:36.029826 11392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:45:36.031510 11392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:36.031826 11392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:36.032140 11392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:36.038128 11406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:36.038182 11402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:36.038301 11392 server_base.cc:1034] running on GCE node
W20260808 00:45:36.038192 11403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:36.038589 11392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:36.038866 11392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:36.040112 11392 hybrid_clock.cc:648] HybridClock initialized: now 1786149936040093 us; error 36 us; skew 500 ppm
Aug 08 00:45:36 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149936, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:36.043327 11392 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:45:36.043955 11392 webserver.cc:460] Webserver started at http://127.20.105.66:45333/ using document root <none> and password file <none>
I20260808 00:45:36.044368 11392 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:36.044451 11392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:36.044621 11392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:36.046236 11392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "c090af398af14974ad700d27b72e4497"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "711fdbb9bccb5bb25432f14134f181a1"
server_key_iv: "b1bd8658f75ef579900c85b7e99c3f1c"
server_key_version: "encryptionkey@0"
I20260808 00:45:36.046675 11392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "c090af398af14974ad700d27b72e4497"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "711fdbb9bccb5bb25432f14134f181a1"
server_key_iv: "b1bd8658f75ef579900c85b7e99c3f1c"
server_key_version: "encryptionkey@0"
I20260808 00:45:36.048452 11392 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:45:36.049557 11413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:36.049788 11392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:36.049912 11392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "c090af398af14974ad700d27b72e4497"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "711fdbb9bccb5bb25432f14134f181a1"
server_key_iv: "b1bd8658f75ef579900c85b7e99c3f1c"
server_key_version: "encryptionkey@0"
I20260808 00:45:36.050033 11392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:36.062166 11392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:36.065280 11392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:36.065500 11392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:36.065814 11392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:36.066220 11392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:36.066277 11392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:36.066339 11392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:36.066370 11392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:36.073846 11392 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:38031
I20260808 00:45:36.073902 11577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:38031 every 8 connection(s)
I20260808 00:45:36.074302 11392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:45:36 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149936, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:36.083781 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 11392
I20260808 00:45:36.083923 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:45:36.084230 20901 external_mini_cluster.cc:1428] Setting key 5b35f19396e171987e18db6b1edbab8b
I20260808 00:45:36.085476 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.075237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:47177 (local address 127.20.105.126:34797)
0808 00:45:36.075309 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.075311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.075545 (+ 234us) server_negotiation.cc:407] Connection header received
0808 00:45:36.076196 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.076198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.076212 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.076262 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:36.077410 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.078026 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.079212 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.079435 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.079518 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.082068 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:36.082070 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:36.082070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:36.082077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:36.083671 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:36.084279 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:36.084280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:36.084280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:36.084355 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:36.084691 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:36.084693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:36.084693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:36.084852 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:36.084921 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.085284 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.085395 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:45:36.085805 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.075194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:47177)
0808 00:45:36.075394 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:36.075439 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:36.076064 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:36.076332 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:36.076336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:36.077017 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:36.077278 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.077285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.078138 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.078139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.079124 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.079127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.079537 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.079539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.079681 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.080337 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:36.080347 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:36.081960 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:36.083752 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:36.083761 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:36.083764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:36.084132 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:36.084445 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:36.084446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:36.084446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:36.084601 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:36.084917 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:36.084925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:36.085126 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 00:45:36.085245 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 00:45:36.085489 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":115,"threads_started":1}
I20260808 00:45:36.086408 11578 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34797
I20260808 00:45:36.086551 11578 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:36.086791 11578 heartbeater.cc:511] Master 127.20.105.126:34797 requested a full tablet report, sending...
I20260808 00:45:36.087227 11105 ts_manager.cc:194] Registered new tserver with Master: c090af398af14974ad700d27b72e4497 (127.20.105.66:38031)
I20260808 00:45:36.087740 11105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:47177
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:45:36.120573 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34797
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:36.220800 11585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:36.221140 11585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:36.221189 11585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:36.222520 11585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:36.222571 11585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:36.222601 11585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:45:36.224339 11585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:36.224650 11585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:36.224951 11585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:36.231418 11597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:36.231518 11594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:36.231436 11595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:36.231642 11585 server_base.cc:1034] running on GCE node
I20260808 00:45:36.231967 11585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:36.232378 11585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:36.233542 11585 hybrid_clock.cc:648] HybridClock initialized: now 1786149936233496 us; error 62 us; skew 500 ppm
Aug 08 00:45:36 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149936, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:36.236683 11585 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:45:36.237259 11585 webserver.cc:460] Webserver started at http://127.20.105.67:39441/ using document root <none> and password file <none>
I20260808 00:45:36.237556 11585 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:36.237607 11585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:36.237782 11585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:45:36.239063 11585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "81b3baf2d1a24709a22033f547a76f95"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "09ffb72d699fb3cf2eb63abaca9f81d9"
server_key_iv: "0d45b46595857fac00ec1479f914ee7b"
server_key_version: "encryptionkey@0"
I20260808 00:45:36.239482 11585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "81b3baf2d1a24709a22033f547a76f95"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "09ffb72d699fb3cf2eb63abaca9f81d9"
server_key_iv: "0d45b46595857fac00ec1479f914ee7b"
server_key_version: "encryptionkey@0"
I20260808 00:45:36.241178 11585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 00:45:36.242062 11605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:36.242261 11585 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:36.242359 11585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "81b3baf2d1a24709a22033f547a76f95"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "09ffb72d699fb3cf2eb63abaca9f81d9"
server_key_iv: "0d45b46595857fac00ec1479f914ee7b"
server_key_version: "encryptionkey@0"
I20260808 00:45:36.242427 11585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:36.266149 11585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:36.268965 11585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:36.269121 11585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:36.269479 11585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:36.269927 11585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:36.269968 11585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:36.269999 11585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:36.270013 11585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:36.277480 11585 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:35827
I20260808 00:45:36.277549 11768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:35827 every 8 connection(s)
I20260808 00:45:36.278013 11585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:45:36.279615 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 11585
I20260808 00:45:36.279728 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:45:36.279948 20901 external_mini_cluster.cc:1428] Setting key 23d59d0743b599e5049c1090e0b5abf3
Aug 08 00:45:36 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149936, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:36.289376 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.279031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:34085 (local address 127.20.105.126:34797)
0808 00:45:36.279129 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.279131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.279418 (+ 287us) server_negotiation.cc:407] Connection header received
0808 00:45:36.280210 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.280211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.280228 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.280295 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:36.281465 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.282042 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.283195 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.283419 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.283513 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.286325 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:36.286327 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:36.286327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:36.286342 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:36.287707 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:36.288245 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:36.288247 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:36.288248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:36.288320 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:36.288598 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:36.288598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:36.288599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:36.288749 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:36.288824 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.289152 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.289262 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:45:36.289623 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.279014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:34085)
0808 00:45:36.279215 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:36.279267 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:36.280079 (+ 812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:36.280389 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:36.280394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:36.281098 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:36.281351 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.281357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.282149 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.282150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.283090 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.283092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.283524 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.283525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.283706 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.284442 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:36.284453 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:36.286203 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:36.287815 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:36.287823 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:36.287827 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:36.288165 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:36.288390 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:36.288391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:36.288391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:36.288523 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:36.288823 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:36.288831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:36.289001 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 00:45:36.289111 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 00:45:36.289309 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
I20260808 00:45:36.290256 11769 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34797
I20260808 00:45:36.290381 11769 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:36.290620 11769 heartbeater.cc:511] Master 127.20.105.126:34797 requested a full tablet report, sending...
I20260808 00:45:36.291121 11105 ts_manager.cc:194] Registered new tserver with Master: 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827)
I20260808 00:45:36.291744 11105 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:34085
I20260808 00:45:36.292063 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:45:36.300554 11163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.293671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47862 (local address 127.20.105.126:34797)
0808 00:45:36.293716 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.293722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.293865 (+ 143us) server_negotiation.cc:407] Connection header received
0808 00:45:36.293958 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.293959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.293969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.294036 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:36.294648 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.295155 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.295907 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.296174 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.296248 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.297724 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:36.297726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:36.297726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:36.297734 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:36.299161 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:36.299545 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:36.299547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:36.299548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:36.299612 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:36.299818 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:36.299819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:36.299820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:36.300082 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:36.300205 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.300314 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.300412 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:45:36.302599 11105 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:45:36.304841 11105 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:45:36.312650 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.308117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:37417 (local address 127.0.0.1:35446)
0808 00:45:36.308675 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:36.308685 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:36.308743 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:36.309012 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:36.309013 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:36.309015 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:36.309060 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.309061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.310355 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.310364 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.311436 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.311438 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.312270 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.312271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.312353 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.312382 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 00:45:36.312484 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 00:45:36.312553 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":100,"threads_started":1}
I20260808 00:45:36.312664 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.308313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.20.105.65:37417)
0808 00:45:36.308518 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.308524 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.308819 (+ 295us) server_negotiation.cc:407] Connection header received
0808 00:45:36.308892 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.308896 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.308915 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.309016 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:36.309119 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.310225 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.311539 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.312189 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.312272 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.312352 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.312510 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.312566 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260808 00:45:36.312870 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.307753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:38031 (local address 127.0.0.1:38604)
0808 00:45:36.308055 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:36.308068 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:36.308545 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:36.308873 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:36.308881 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:36.308884 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:36.308949 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.308952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.310644 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.310645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.311630 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.311634 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.312663 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.312664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.312725 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.312729 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:45:36.312764 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 00:45:36.312799 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":27,"thread_start_us":125,"threads_started":1}
I20260808 00:45:36.313030 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.307753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:35827 (local address 127.0.0.1:33482)
0808 00:45:36.307962 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:36.308061 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:36.308152 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:36.308700 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:36.308704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:36.308708 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:36.308820 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.308823 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.309979 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.309980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.311326 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.311329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.312423 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.312424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.312514 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.312518 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:45:36.312581 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:45:36.312641 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":124,"threads_started":1}
I20260808 00:45:36.313246 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.308267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.20.105.67:35827)
0808 00:45:36.308470 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.308476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.308488 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:45:36.308540 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.308543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.308563 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.308907 (+ 344us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:36.309004 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.309874 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.311431 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.312242 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.312322 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.312380 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.312726 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.312781 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":121,"threads_started":1}
I20260808 00:45:36.313287 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.307974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.20.105.66:38031)
0808 00:45:36.308675 (+ 701us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.308679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.308690 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:45:36.308733 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.308735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.308751 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.308885 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:36.309058 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.310507 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.311752 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.312565 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.313026 (+ 461us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.313100 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.313130 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.313184 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":147,"threads_started":1}
I20260808 00:45:36.314168 11480 tablet_service.cc:1467] Processing CreateTablet for tablet 6abab42e6bf643ad8e5989ff330a174b (DEFAULT_TABLE table=test-table [id=290dbf29cc1949e1aca319a9bb8d574e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:36.314168 11674 tablet_service.cc:1467] Processing CreateTablet for tablet 6abab42e6bf643ad8e5989ff330a174b (DEFAULT_TABLE table=test-table [id=290dbf29cc1949e1aca319a9bb8d574e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:36.314234 11291 tablet_service.cc:1467] Processing CreateTablet for tablet 6abab42e6bf643ad8e5989ff330a174b (DEFAULT_TABLE table=test-table [id=290dbf29cc1949e1aca319a9bb8d574e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:36.314653 11291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6abab42e6bf643ad8e5989ff330a174b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:36.314652 11480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6abab42e6bf643ad8e5989ff330a174b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:36.314652 11674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6abab42e6bf643ad8e5989ff330a174b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:36.318277 11790 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Bootstrap starting.
I20260808 00:45:36.318691 11791 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Bootstrap starting.
I20260808 00:45:36.318737 11792 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: Bootstrap starting.
I20260808 00:45:36.319522 11792 tablet_bootstrap.cc:654] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:36.319522 11790 tablet_bootstrap.cc:654] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:36.319538 11791 tablet_bootstrap.cc:654] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:36.320303 11792 log.cc:824] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:36.320290 11791 log.cc:824] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:36.320668 11790 log.cc:824] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:36.321244 11791 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: No bootstrap required, opened a new log
I20260808 00:45:36.321255 11792 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: No bootstrap required, opened a new log
I20260808 00:45:36.321326 11792 ts_tablet_manager.cc:1397] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:45:36.321434 11791 ts_tablet_manager.cc:1397] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260808 00:45:36.321568 11790 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: No bootstrap required, opened a new log
I20260808 00:45:36.321656 11790 ts_tablet_manager.cc:1397] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260808 00:45:36.323200 11790 raft_consensus.cc:348] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.323421 11790 raft_consensus.cc:374] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:36.323475 11790 raft_consensus.cc:729] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3de2302cb84028b2b60596fe36e229, State: Initialized, Role: FOLLOWER
I20260808 00:45:36.323627 11790 consensus_queue.cc:260] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.323587 11792 raft_consensus.cc:348] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.323650 11791 raft_consensus.cc:348] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.323786 11792 raft_consensus.cc:374] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:36.323802 11791 raft_consensus.cc:374] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:36.323824 11792 raft_consensus.cc:729] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b3baf2d1a24709a22033f547a76f95, State: Initialized, Role: FOLLOWER
I20260808 00:45:36.323836 11791 raft_consensus.cc:729] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c090af398af14974ad700d27b72e4497, State: Initialized, Role: FOLLOWER
I20260808 00:45:36.323999 11792 consensus_queue.cc:260] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.323993 11791 consensus_queue.cc:260] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.324013 11790 ts_tablet_manager.cc:1428] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:45:36.324216 11385 heartbeater.cc:503] Master 127.20.105.126:34797 was elected leader, sending a full tablet report...
I20260808 00:45:36.324327 11791 ts_tablet_manager.cc:1428] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 00:45:36.324327 11792 ts_tablet_manager.cc:1428] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:45:36.324451 11578 heartbeater.cc:503] Master 127.20.105.126:34797 was elected leader, sending a full tablet report...
I20260808 00:45:36.324582 11769 heartbeater.cc:503] Master 127.20.105.126:34797 was elected leader, sending a full tablet report...
W20260808 00:45:36.324975 11579 tablet.cc:2369] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:45:36.367616 11804 raft_consensus.cc:482] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:36.367810 11804 raft_consensus.cc:504] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.368461 11804 leader_election.cc:288] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827), fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417)
I20260808 00:45:36.373195 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.368833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:42631 (local address 127.20.105.67:35827)
0808 00:45:36.368896 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.368899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.368910 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:45:36.368949 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.368950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.368962 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.369034 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:36.369379 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.370269 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.371959 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.372718 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.372795 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.372806 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.373052 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.373108 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:45:36.373195 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.368801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:56751 (local address 127.20.105.65:37417)
0808 00:45:36.368862 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.368865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.369172 (+ 307us) server_negotiation.cc:407] Connection header received
0808 00:45:36.369362 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.369364 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.369379 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.369456 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:36.369698 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.370469 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.371981 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.372672 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.372781 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.372795 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.373054 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.373111 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:45:36.373195 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.368673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:35827 (local address 127.20.105.66:42631)
0808 00:45:36.368744 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:36.368762 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:36.368831 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:36.369079 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:36.369080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:36.369083 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:36.369260 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.369264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.370461 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.370462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.371830 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.371833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.372824 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.372826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.372933 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.372935 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:45:36.373005 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:45:36.373072 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":19}
I20260808 00:45:36.373202 11806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.368676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:37417 (local address 127.20.105.66:56751)
0808 00:45:36.369043 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:36.369055 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:36.369185 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:36.369480 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:36.369480 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:36.369483 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:36.369597 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.369600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.370548 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.370548 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.371830 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:36.371833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.372824 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.372826 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:36.372925 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.372930 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:45:36.373005 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:45:36.373072 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":109,"thread_start_us":166,"threads_started":1}
I20260808 00:45:36.373798 11704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6abab42e6bf643ad8e5989ff330a174b" candidate_uuid: "c090af398af14974ad700d27b72e4497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81b3baf2d1a24709a22033f547a76f95" is_pre_election: true
I20260808 00:45:36.373929 11326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6abab42e6bf643ad8e5989ff330a174b" candidate_uuid: "c090af398af14974ad700d27b72e4497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3de2302cb84028b2b60596fe36e229" is_pre_election: true
I20260808 00:45:36.374038 11704 raft_consensus.cc:2454] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c090af398af14974ad700d27b72e4497 in term 0.
I20260808 00:45:36.374169 11326 raft_consensus.cc:2454] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c090af398af14974ad700d27b72e4497 in term 0.
I20260808 00:45:36.374402 11420 leader_election.cc:302] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497; no voters:
I20260808 00:45:36.374593 11804 raft_consensus.cc:2781] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:45:36.374660 11804 raft_consensus.cc:482] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:45:36.374688 11804 raft_consensus.cc:3037] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:36.375605 11804 raft_consensus.cc:504] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.375789 11804 leader_election.cc:288] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [CANDIDATE]: Term 1 election: Requested vote from peers 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827), fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417)
I20260808 00:45:36.375972 11326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6abab42e6bf643ad8e5989ff330a174b" candidate_uuid: "c090af398af14974ad700d27b72e4497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3de2302cb84028b2b60596fe36e229"
I20260808 00:45:36.375972 11704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6abab42e6bf643ad8e5989ff330a174b" candidate_uuid: "c090af398af14974ad700d27b72e4497" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81b3baf2d1a24709a22033f547a76f95"
I20260808 00:45:36.376109 11704 raft_consensus.cc:3037] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:36.376106 11326 raft_consensus.cc:3037] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:36.376999 11704 raft_consensus.cc:2454] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c090af398af14974ad700d27b72e4497 in term 1.
I20260808 00:45:36.377084 11326 raft_consensus.cc:2454] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c090af398af14974ad700d27b72e4497 in term 1.
I20260808 00:45:36.377185 11420 leader_election.cc:302] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497; no voters:
I20260808 00:45:36.377302 11804 raft_consensus.cc:2781] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:36.377511 11804 raft_consensus.cc:686] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 1 LEADER]: Becoming Leader. State: Replica: c090af398af14974ad700d27b72e4497, State: Running, Role: LEADER
I20260808 00:45:36.377671 11804 consensus_queue.cc:237] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:36.378924 11105 catalog_manager.cc:5495] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 reported cstate change: term changed from 0 to 1, leader changed from <none> to c090af398af14974ad700d27b72e4497 (127.20.105.66). New cstate: current_term: 1 leader_uuid: "c090af398af14974ad700d27b72e4497" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } health_report { overall_health: UNKNOWN } } }
W20260808 00:45:36.384698 11386 tablet.cc:2369] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:45:36.387037 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.384525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38614 (local address 127.20.105.66:38031)
0808 00:45:36.384579 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:36.384582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:36.384593 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:45:36.384640 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:36.384641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:36.384665 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:36.384752 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:45:36.384956 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.385498 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.386302 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:36.386497 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:36.386554 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:36.386676 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:45:36.386796 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:45:36.386860 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:36.386875 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 00:45:36.386895 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:45:36.389467 11326 raft_consensus.cc:1261] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Refusing update from remote peer c090af398af14974ad700d27b72e4497: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:36.389473 11704 raft_consensus.cc:1261] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 1 FOLLOWER]: Refusing update from remote peer c090af398af14974ad700d27b72e4497: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:36.389955 11807 consensus_queue.cc:1035] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:36.390120 11804 consensus_queue.cc:1035] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:36.393723 11819 mvcc.cc:205] Tried to move back new op lower bound from 7316070139447857152 to 7316070139403866112. Current Snapshot: MvccSnapshot[applied={T|T < 7316070139447857152}]
I20260808 00:45:36.394554 11814 mvcc.cc:205] Tried to move back new op lower bound from 7316070139447857152 to 7316070139403866112. Current Snapshot: MvccSnapshot[applied={T|T < 7316070139447857152 or (T in {7316070139447857152})}]
I20260808 00:45:36.396474 11105 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:36.396589 11105 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:36.397632 11105 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Sending DeleteTablet for 3 replicas of tablet 6abab42e6bf643ad8e5989ff330a174b
I20260808 00:45:36.398087 11480 tablet_service.cc:1514] Processing DeleteTablet for tablet 6abab42e6bf643ad8e5989ff330a174b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:36 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:38604
I20260808 00:45:36.398132 11674 tablet_service.cc:1514] Processing DeleteTablet for tablet 6abab42e6bf643ad8e5989ff330a174b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:36 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:33482
I20260808 00:45:36.398392 11832 tablet_replica.cc:330] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: stopping tablet replica
I20260808 00:45:36.398417 11291 tablet_service.cc:1514] Processing DeleteTablet for tablet 6abab42e6bf643ad8e5989ff330a174b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:36 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:35446
I20260808 00:45:36.398530 11832 raft_consensus.cc:2229] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:36.398680 11832 raft_consensus.cc:2258] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:36.398674 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 11198
I20260808 00:45:36.398675 11834 tablet_replica.cc:330] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: stopping tablet replica
I20260808 00:45:36.398910 11834 raft_consensus.cc:2229] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:36.399109 11834 raft_consensus.cc:2258] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:36.399271 11832 ts_tablet_manager.cc:1905] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:36.402076 11832 ts_tablet_manager.cc:1918] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:36.402190 11832 log.cc:1196] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/6abab42e6bf643ad8e5989ff330a174b
I20260808 00:45:36.402565 11832 ts_tablet_manager.cc:1939] T 6abab42e6bf643ad8e5989ff330a174b P 81b3baf2d1a24709a22033f547a76f95: Deleting consensus metadata
I20260808 00:45:36.403283 11090 catalog_manager.cc:4841] TS 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827): tablet 6abab42e6bf643ad8e5989ff330a174b (table test-table [id=290dbf29cc1949e1aca319a9bb8d574e]) successfully deleted
I20260808 00:45:36.409755 11831 tablet_replica.cc:330] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: stopping tablet replica
I20260808 00:45:36.410135 11831 raft_consensus.cc:2229] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:45:36.410378 11831 raft_consensus.cc:2258] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:36.412667 11831 ts_tablet_manager.cc:1905] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:36.414820 11831 ts_tablet_manager.cc:1918] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:36.414933 11831 log.cc:1196] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/6abab42e6bf643ad8e5989ff330a174b
I20260808 00:45:36.415261 11831 ts_tablet_manager.cc:1939] T 6abab42e6bf643ad8e5989ff330a174b P c090af398af14974ad700d27b72e4497: Deleting consensus metadata
I20260808 00:45:36.415939 11083 catalog_manager.cc:4841] TS c090af398af14974ad700d27b72e4497 (127.20.105.66:38031): tablet 6abab42e6bf643ad8e5989ff330a174b (table test-table [id=290dbf29cc1949e1aca319a9bb8d574e]) successfully deleted
W20260808 00:45:36.419029 11083 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:37417 (error 108)
I20260808 00:45:36.419368 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 11392
I20260808 00:45:36.433343 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:36.433080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:37417 (local address 127.0.0.1:35456)
0808 00:45:36.433178 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:36.433260 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 00:45:36.433807 11083 catalog_manager.cc:4584] TS fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6abab42e6bf643ad8e5989ff330a174b: Network error: Client connection negotiation failed: client connection to 127.20.105.65:37417: connect: Connection refused (error 111)
I20260808 00:45:36.446508 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 11585
I20260808 00:45:36.458559 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 11059
I20260808 00:45:56.473992 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34797
--webserver_interface=127.20.105.126
--webserver_port=42823
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:34797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:45:56.571115 11838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:56.571452 11838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:56.571497 11838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:56.572830 11838 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:45:56.572908 11838 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:45:56.572932 11838 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:45:56.572949 11838 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:45:56.572968 11838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:56.572986 11838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:56.573007 11838 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:45:56.573029 11838 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:45:56.574898 11838 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:34797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=42823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:56.575379 11838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:56.575716 11838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:56.588090 11848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:56.588104 11850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:56.588168 11846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:56.588297 11838 server_base.cc:1034] running on GCE node
I20260808 00:45:56.588742 11838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:56.589197 11838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:56.590430 11838 hybrid_clock.cc:648] HybridClock initialized: now 1786149956590403 us; error 41 us; skew 500 ppm
Aug 08 00:45:56 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:56.598438 11838 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:45:56.599138 11838 webserver.cc:460] Webserver started at http://127.20.105.126:42823/ using document root <none> and password file <none>
I20260808 00:45:56.599458 11838 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:56.599565 11838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:56.601373 11838 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:56.602214 11864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:56.602541 11838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:56.602680 11838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "171543b6fa974aa0a9919280b50a9b9f"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "33087359d923cd7723077f15818d3cd6"
server_key_iv: "9b9e26ca745b3f4c592e13e8152f92fb"
server_key_version: "encryptionkey@0"
I20260808 00:45:56.603237 11838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:56.610718 11838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:56.613417 11838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:56.613605 11838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:56.618999 11838 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:34797
I20260808 00:45:56.619040 11940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:34797 every 8 connection(s)
I20260808 00:45:56.619534 11838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:45:56.621609 11942 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:45:56.622499 11942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Bootstrap starting.
I20260808 00:45:56.622857 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 11838
I20260808 00:45:56.623291 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:37417
--local_ip_for_outbound_sockets=127.20.105.65
--tserver_master_addrs=127.20.105.126:34797
--webserver_port=42329
--webserver_interface=127.20.105.65
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:56.635677 11942 log.cc:824] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:56.638517 11942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:45:56.638981 11942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Bootstrap complete.
I20260808 00:45:56.641973 11942 raft_consensus.cc:348] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:45:56.642295 11942 raft_consensus.cc:729] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 171543b6fa974aa0a9919280b50a9b9f, State: Initialized, Role: FOLLOWER
I20260808 00:45:56.642518 11942 consensus_queue.cc:260] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:45:56.642604 11942 raft_consensus.cc:388] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:45:56.642683 11942 raft_consensus.cc:482] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:45:56.642738 11942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:45:56.644191 11942 raft_consensus.cc:504] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:45:56.644328 11942 leader_election.cc:302] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 171543b6fa974aa0a9919280b50a9b9f; no voters:
I20260808 00:45:56.644582 11942 leader_election.cc:288] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:45:56.644632 11946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:45:56.644843 11946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 2 LEADER]: Becoming Leader. State: Replica: 171543b6fa974aa0a9919280b50a9b9f, State: Running, Role: LEADER
I20260808 00:45:56.644972 11942 sys_catalog.cc:564] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:45:56.645008 11946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:45:56.645480 11946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 171543b6fa974aa0a9919280b50a9b9f. Latest consensus state: current_term: 2 leader_uuid: "171543b6fa974aa0a9919280b50a9b9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } } }
I20260808 00:45:56.645481 11950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "171543b6fa974aa0a9919280b50a9b9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } } }
I20260808 00:45:56.646136 11946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:56.646174 11950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: This master's current role is: LEADER
I20260808 00:45:56.646468 11966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:45:56.647646 11966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:45:56.647848 11966 catalog_manager.cc:1259] Loaded cluster ID: c38bf853828d4318b9a4f6ed8ef6bd68
I20260808 00:45:56.648145 11966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:45:56.649107 11966 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:45:56.649356 11966 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Loaded TSK: 0
I20260808 00:45:56.658125 11966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Generated new TSK 1
I20260808 00:45:56.658268 11966 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:45:56.735062 11944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:56.735355 11944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:56.735396 11944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:56.736725 11944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:56.736800 11944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:56.736848 11944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:45:56.738453 11944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:37417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=42329
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:56.738775 11944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:56.739070 11944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:56.745471 11983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:56.745615 11944 server_base.cc:1034] running on GCE node
W20260808 00:45:56.745343 11982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:56.745347 11985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:56.746018 11944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:56.746373 11944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:56.747622 11944 hybrid_clock.cc:648] HybridClock initialized: now 1786149956747602 us; error 35 us; skew 500 ppm
Aug 08 00:45:56 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:56.751202 11944 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:45:56.751932 11944 webserver.cc:460] Webserver started at http://127.20.105.65:42329/ using document root <none> and password file <none>
I20260808 00:45:56.752345 11944 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:56.752465 11944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:56.754453 11944 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:45:56.755615 11995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:56.758885 11944 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260808 00:45:56.759079 11944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "fa3de2302cb84028b2b60596fe36e229"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "e818fb19683185a8f0c45839a39719e0"
server_key_iv: "071331a0ba3aa3a2b7b8faeff5a142f6"
server_key_version: "encryptionkey@0"
I20260808 00:45:56.759791 11944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:56.781574 11944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:56.784466 11944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:56.784659 11944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:56.784981 11944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:56.785580 12003 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:45:56.787374 11944 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:45:56.787431 11944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 00:45:56.787462 11944 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:45:56.788383 11944 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:45:56.788429 11944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:56.788483 12003 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Bootstrap starting.
I20260808 00:45:56.798228 11944 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:37417
I20260808 00:45:56.798916 11944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:45:56.799789 12151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:37417 every 8 connection(s)
I20260808 00:45:56.802384 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 11944
I20260808 00:45:56.802726 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:38031
--local_ip_for_outbound_sockets=127.20.105.66
--tserver_master_addrs=127.20.105.126:34797
--webserver_port=45333
--webserver_interface=127.20.105.66
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:45:56.805449 12003 log.cc:824] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Log is configured to *not* fsync() on all Append() calls
Aug 08 00:45:56 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:56.811419 12003 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:45:56.811833 12003 tablet_bootstrap.cc:492] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Bootstrap complete.
I20260808 00:45:56.813328 12003 ts_tablet_manager.cc:1397] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Time spent bootstrapping tablet: real 0.025s user 0.000s sys 0.019s
I20260808 00:45:56.814968 12003 raft_consensus.cc:348] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:56.815341 12003 raft_consensus.cc:729] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3de2302cb84028b2b60596fe36e229, State: Initialized, Role: FOLLOWER
I20260808 00:45:56.815515 12003 consensus_queue.cc:260] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:56.815922 12003 ts_tablet_manager.cc:1428] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20260808 00:45:56.816181 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:56.800274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:56729 (local address 127.20.105.126:34797)
0808 00:45:56.800567 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:56.800578 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:56.800891 (+ 313us) server_negotiation.cc:407] Connection header received
0808 00:45:56.801747 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:56.801757 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:56.801836 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:56.802060 (+ 224us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:56.803142 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.804483 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.806095 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.806799 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.806916 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:56.810313 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:56.810319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:56.810319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:56.810335 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:56.813188 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:56.813956 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:56.813960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:56.813961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:56.814168 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:56.814547 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:56.814548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:56.814549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:56.814922 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:56.815026 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:56.815556 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 00:45:56.815785 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":104,"threads_started":1}
I20260808 00:45:56.816514 12155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:56.799847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:56729)
0808 00:45:56.800676 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:56.800727 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:56.801463 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:56.802102 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:56.802108 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:56.802607 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:56.802963 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:56.802972 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.804658 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.804660 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:56.805993 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:56.805996 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.806930 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.806933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:56.807117 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:56.807880 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:56.807893 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:56.810168 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:56.813310 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:56.813321 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:56.813324 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:56.813819 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:56.814264 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:56.814265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:56.814266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:56.814453 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:56.815019 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:56.815029 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:56.815272 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 00:45:56.816092 (+ 820us) client_negotiation.cc:240] Negotiation successful
0808 00:45:56.816316 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":100,"threads_started":1}
I20260808 00:45:56.817391 12153 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34797
I20260808 00:45:56.817495 12153 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:56.817734 12153 heartbeater.cc:511] Master 127.20.105.126:34797 requested a full tablet report, sending...
I20260808 00:45:56.818701 11888 ts_manager.cc:194] Registered new tserver with Master: fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417)
I20260808 00:45:56.820451 11888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:56729
I20260808 00:45:56.821527 12153 heartbeater.cc:503] Master 127.20.105.126:34797 was elected leader, sending a full tablet report...
Aug 08 00:45:56 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.65@KRBTEST.COM
I20260808 00:45:56.830338 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:56.819634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:37417 (local address 127.0.0.1:37936)
0808 00:45:56.819954 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:56.820001 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:56.820116 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:56.820382 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:56.820388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:56.820854 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:56.820943 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:56.820952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.822707 (+ 1755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.822708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:56.823398 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:56.823401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.823985 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.823986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:56.824116 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:56.824769 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:56.824775 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:56.826727 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:56.828924 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:56.828934 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:56.828936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:56.829180 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:56.829436 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:56.829437 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:56.829438 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:56.829509 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:56.829844 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:56.829852 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:56.829925 (+ 73us) client_negotiation.cc:769] Sending connection context
0808 00:45:56.830094 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 00:45:56.830244 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":146,"threads_started":1}
I20260808 00:45:56.830456 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:56.819719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37936 (local address 127.20.105.65:37417)
0808 00:45:56.820066 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:56.820072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:56.820085 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:45:56.820250 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:56.820253 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:56.820271 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:56.820378 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:56.821703 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.822607 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.823525 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.823872 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.823983 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:56.826860 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:56.826871 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:56.826872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:56.826881 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:56.828802 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:56.829278 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:56.829282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:56.829282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:56.829355 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:56.829583 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:56.829584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:56.829584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:56.829765 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:56.829872 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:56.830185 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 00:45:56.830333 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":87,"threads_started":1}
I20260808 00:45:56.831043 12055 tablet_service.cc:1514] Processing DeleteTablet for tablet 6abab42e6bf643ad8e5989ff330a174b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:36 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37936
I20260808 00:45:56.831930 12166 tablet_replica.cc:330] stopping tablet replica
I20260808 00:45:56.832132 12166 raft_consensus.cc:2229] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:56.832252 12166 raft_consensus.cc:2258] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:56.832687 12166 ts_tablet_manager.cc:1905] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:56.835187 12166 ts_tablet_manager.cc:1918] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:56.835265 12166 log.cc:1196] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal/wals/6abab42e6bf643ad8e5989ff330a174b
I20260808 00:45:56.835579 12166 ts_tablet_manager.cc:1939] T 6abab42e6bf643ad8e5989ff330a174b P fa3de2302cb84028b2b60596fe36e229: Deleting consensus metadata
I20260808 00:45:56.836297 11865 catalog_manager.cc:4841] TS fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417): tablet 6abab42e6bf643ad8e5989ff330a174b (table test-table [id=290dbf29cc1949e1aca319a9bb8d574e]) successfully deleted
W20260808 00:45:56.910745 12157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:56.911052 12157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:56.911093 12157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:56.912376 12157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:56.912463 12157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:56.912510 12157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:45:56.914199 12157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:38031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=45333
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:56.914534 12157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:56.914825 12157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:56.920758 12172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:56.920758 12174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:56.920802 12171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:56.921176 12157 server_base.cc:1034] running on GCE node
I20260808 00:45:56.921381 12157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:56.921689 12157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:56.922853 12157 hybrid_clock.cc:648] HybridClock initialized: now 1786149956922816 us; error 55 us; skew 500 ppm
Aug 08 00:45:56 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:56.925925 12157 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:45:56.926417 12157 webserver.cc:460] Webserver started at http://127.20.105.66:45333/ using document root <none> and password file <none>
I20260808 00:45:56.926697 12157 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:56.926764 12157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:56.928471 12157 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:45:56.929306 12188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:56.929596 12157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:56.929728 12157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "c090af398af14974ad700d27b72e4497"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "711fdbb9bccb5bb25432f14134f181a1"
server_key_iv: "b1bd8658f75ef579900c85b7e99c3f1c"
server_key_version: "encryptionkey@0"
I20260808 00:45:56.930188 12157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:56.949397 12157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:56.952351 12157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:56.952553 12157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:56.952864 12157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:56.953250 12157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:56.953310 12157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:56.953361 12157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:56.953392 12157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:56.960467 12157 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:38031
I20260808 00:45:56.960547 12354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:38031 every 8 connection(s)
I20260808 00:45:56.961043 12157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:45:56.962513 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 12157
I20260808 00:45:56.962829 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:35827
--local_ip_for_outbound_sockets=127.20.105.67
--tserver_master_addrs=127.20.105.126:34797
--webserver_port=39441
--webserver_interface=127.20.105.67
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:45:56 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149956, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:56.974205 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:56.962028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:55953 (local address 127.20.105.126:34797)
0808 00:45:56.962141 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:56.962144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:56.962395 (+ 251us) server_negotiation.cc:407] Connection header received
0808 00:45:56.963231 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:56.963233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:56.963257 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:56.963335 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:56.964764 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.965730 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.966876 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.967264 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.967354 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:56.969992 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:56.969994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:56.969994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:56.970007 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:56.972199 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:56.972721 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:56.972723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:56.972724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:56.972815 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:56.973160 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:56.973161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:56.973161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:56.973372 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:56.973469 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:56.973852 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 00:45:56.974010 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 00:45:56.974310 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:56.961996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:55953)
0808 00:45:56.962220 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:56.962268 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:56.963079 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:56.963444 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:56.963449 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:56.964307 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:56.964623 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:56.964631 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.965851 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.965852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:56.966758 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:56.966761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:56.967365 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:56.967366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:56.967496 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:56.968178 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:56.968188 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:56.969860 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:56.972320 (+ 2460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:56.972327 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:56.972330 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:56.972628 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:56.972926 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:56.972927 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:56.972927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:56.973073 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:56.973489 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:56.973498 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:56.973670 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 00:45:56.973793 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 00:45:56.974008 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":111,"threads_started":1}
I20260808 00:45:56.975304 12357 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34797
I20260808 00:45:56.975481 12357 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:56.975764 12357 heartbeater.cc:511] Master 127.20.105.126:34797 requested a full tablet report, sending...
I20260808 00:45:56.976459 11888 ts_manager.cc:194] Registered new tserver with Master: c090af398af14974ad700d27b72e4497 (127.20.105.66:38031)
I20260808 00:45:56.977195 11888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:55953
W20260808 00:45:57.066468 12360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:45:57.066746 12360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:45:57.066773 12360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:45:57.068070 12360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:45:57.068116 12360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:45:57.068145 12360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:45:57.069772 12360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:35827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=39441
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:45:57.070035 12360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:45:57.070281 12360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:45:57.076452 12367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:57.076421 12366 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:45:57.076429 12369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:45:57.076665 12360 server_base.cc:1034] running on GCE node
I20260808 00:45:57.076996 12360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:45:57.077337 12360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:45:57.078503 12360 hybrid_clock.cc:648] HybridClock initialized: now 1786149957078487 us; error 36 us; skew 500 ppm
Aug 08 00:45:57 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149957, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:45:57.081629 12360 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:45:57.082137 12360 webserver.cc:460] Webserver started at http://127.20.105.67:39441/ using document root <none> and password file <none>
I20260808 00:45:57.082453 12360 fs_manager.cc:359] Metadata directory not provided
I20260808 00:45:57.082559 12360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:45:57.084352 12360 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:45:57.085201 12378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:57.085439 12360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:45:57.085585 12360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "81b3baf2d1a24709a22033f547a76f95"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "09ffb72d699fb3cf2eb63abaca9f81d9"
server_key_iv: "0d45b46595857fac00ec1479f914ee7b"
server_key_version: "encryptionkey@0"
I20260808 00:45:57.086058 12360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:45:57.099773 12360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:45:57.102792 12360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:45:57.103039 12360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:45:57.103372 12360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:45:57.103789 12360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:45:57.103852 12360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:57.103907 12360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:45:57.103941 12360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:45:57.112123 12360 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:35827
I20260808 00:45:57.112179 12545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:35827 every 8 connection(s)
I20260808 00:45:57.112663 12360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:45:57 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149957, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:45:57.122011 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 12360
I20260808 00:45:57.123863 12156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.113618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:44331 (local address 127.20.105.126:34797)
0808 00:45:57.113720 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.113723 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.113974 (+ 251us) server_negotiation.cc:407] Connection header received
0808 00:45:57.114569 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.114570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.114583 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.114642 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:57.115773 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.116401 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.117530 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.117743 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.117806 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.120415 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:57.120416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:57.120417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:57.120424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:57.122108 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:57.122723 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:57.122725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:57.122726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:57.122797 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:57.123090 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:57.123091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:57.123091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:57.123229 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:57.123310 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:57.123635 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 00:45:57.123742 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 00:45:57.124107 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.113599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:44331)
0808 00:45:57.113812 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:57.113857 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:57.114458 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:57.114741 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:57.114746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:57.115436 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:45:57.115675 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.115681 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.116496 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.116497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.117432 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.117436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.117859 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.117860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.118002 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.118559 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:45:57.118569 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:45:57.120311 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:45:57.122250 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:57.122258 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:57.122261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:57.122651 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:57.122890 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:45:57.122890 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:45:57.122891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:45:57.123016 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:45:57.123313 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:45:57.123320 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:45:57.123492 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 00:45:57.123611 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 00:45:57.123799 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":121,"threads_started":1}
I20260808 00:45:57.124644 12546 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34797
I20260808 00:45:57.124782 12546 heartbeater.cc:464] Registering TS with master...
I20260808 00:45:57.125007 12546 heartbeater.cc:511] Master 127.20.105.126:34797 requested a full tablet report, sending...
I20260808 00:45:57.125501 11888 ts_manager.cc:194] Registered new tserver with Master: 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827)
I20260808 00:45:57.125902 11888 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:44331
I20260808 00:45:57.130074 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.122565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.20.105.126:34797)
0808 00:45:57.122721 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.122723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.122820 (+ 97us) server_negotiation.cc:407] Connection header received
0808 00:45:57.122941 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.122942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.122958 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.123055 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:57.123723 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.124517 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.125448 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.125726 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.125803 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.127167 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:57.127169 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:57.127169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:57.127176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:57.128743 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:57.129130 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:57.129131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:57.129131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:57.129202 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:57.129429 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:57.129430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:57.129430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:57.129560 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:57.129806 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:57.129846 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 00:45:57.129949 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
I20260808 00:45:57.131068 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:45:57.134317 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.131543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52248 (local address 127.20.105.126:34797)
0808 00:45:57.131616 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.131618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.131794 (+ 176us) server_negotiation.cc:407] Connection header received
0808 00:45:57.131836 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.131837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.131905 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.131978 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:45:57.132189 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.132678 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.133610 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.133818 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.133885 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.134067 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:45:57.134124 (+ 57us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 00:45:57.134264 (+ 140us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52248: token expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 00:45:57.134398 12551 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52248: token expired
W20260808 00:45:57.134689 11772 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.105.126:34797)
I20260808 00:45:57.141455 12551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.134972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.20.105.126:34797)
0808 00:45:57.135008 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.135009 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.135020 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:45:57.135075 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.135076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.135083 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.135128 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:45:57.135644 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.136189 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.136996 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.137184 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.137247 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.138625 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:45:57.138626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:45:57.138627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:45:57.138633 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:45:57.139997 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:57.140413 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:57.140414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:57.140415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:57.140477 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:45:57.140815 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:45:57.140816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:45:57.140817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:45:57.140958 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:45:57.141073 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:57.141212 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 00:45:57.141333 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:45:57.143010 11888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:45:57.145284 11888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:45:57.148324 12055 tablet_service.cc:1467] Processing CreateTablet for tablet 3e11c945e7e7483b8876b1dd619b9ae3 (DEFAULT_TABLE table=test-table [id=d2948be73cda4adc886b2277ee27a80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:57.148653 12055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e11c945e7e7483b8876b1dd619b9ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:57.150607 12003 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Bootstrap starting.
I20260808 00:45:57.151011 12003 tablet_bootstrap.cc:654] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:57.151849 12003 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: No bootstrap required, opened a new log
I20260808 00:45:57.151913 12003 ts_tablet_manager.cc:1397] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:57.152412 12003 raft_consensus.cc:348] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.152525 12003 raft_consensus.cc:374] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:57.152585 12003 raft_consensus.cc:729] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3de2302cb84028b2b60596fe36e229, State: Initialized, Role: FOLLOWER
I20260808 00:45:57.152644 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.148284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34916 (local address 127.20.105.66:38031)
0808 00:45:57.148528 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.148535 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.148548 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:45:57.148605 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.148607 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.148626 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.148739 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:57.148913 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.150009 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.151249 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.152129 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.152238 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.152301 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:57.152489 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 00:45:57.152537 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":102,"threads_started":1}
I20260808 00:45:57.152639 12163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.147874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:38031 (local address 127.0.0.1:34916)
0808 00:45:57.147930 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:57.147945 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:57.147989 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:57.148768 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:57.148770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:57.148772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:57.148844 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.148846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.150104 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.150105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.151182 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.151185 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.152242 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.152243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.152348 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.152380 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 00:45:57.152456 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 00:45:57.152525 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":3}
I20260808 00:45:57.152644 12003 consensus_queue.cc:260] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.152751 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.148203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.20.105.67:35827)
0808 00:45:57.148576 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.148581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.148594 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:45:57.148669 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.148672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.148689 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.148780 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:57.148996 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.149734 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.151226 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.152117 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.152204 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.152292 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:57.152608 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 00:45:57.152655 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":138,"threads_started":1}
I20260808 00:45:57.152757 12003 ts_tablet_manager.cc:1428] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:45:57.153213 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.147932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:35827 (local address 127.0.0.1:58518)
0808 00:45:57.148504 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:57.148518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:57.148588 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:57.148883 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:57.148884 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:57.148887 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:57.148929 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.148930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.149832 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.149834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.151109 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.151112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.152334 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.152335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.152403 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.152408 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:45:57.152463 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:45:57.152522 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":25,"thread_start_us":496,"threads_started":1}
I20260808 00:45:57.154016 12255 tablet_service.cc:1467] Processing CreateTablet for tablet 3e11c945e7e7483b8876b1dd619b9ae3 (DEFAULT_TABLE table=test-table [id=d2948be73cda4adc886b2277ee27a80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:57.154012 12443 tablet_service.cc:1467] Processing CreateTablet for tablet 3e11c945e7e7483b8876b1dd619b9ae3 (DEFAULT_TABLE table=test-table [id=d2948be73cda4adc886b2277ee27a80d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:45:57.154418 12255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e11c945e7e7483b8876b1dd619b9ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:57.154418 12443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e11c945e7e7483b8876b1dd619b9ae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:45:57.158028 12559 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Bootstrap starting.
I20260808 00:45:57.158037 12560 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Bootstrap starting.
I20260808 00:45:57.158728 12560 tablet_bootstrap.cc:654] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:57.158792 12559 tablet_bootstrap.cc:654] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Neither blocks nor log segments found. Creating new log.
I20260808 00:45:57.159407 12560 log.cc:824] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:57.159407 12559 log.cc:824] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Log is configured to *not* fsync() on all Append() calls
I20260808 00:45:57.160295 12560 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: No bootstrap required, opened a new log
I20260808 00:45:57.160286 12559 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: No bootstrap required, opened a new log
I20260808 00:45:57.160391 12559 ts_tablet_manager.cc:1397] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:45:57.160485 12560 ts_tablet_manager.cc:1397] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:45:57.162009 12560 raft_consensus.cc:348] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.162009 12559 raft_consensus.cc:348] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.162176 12560 raft_consensus.cc:374] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:57.162202 12560 raft_consensus.cc:729] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81b3baf2d1a24709a22033f547a76f95, State: Initialized, Role: FOLLOWER
I20260808 00:45:57.162204 12559 raft_consensus.cc:374] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:45:57.162248 12559 raft_consensus.cc:729] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c090af398af14974ad700d27b72e4497, State: Initialized, Role: FOLLOWER
I20260808 00:45:57.162359 12560 consensus_queue.cc:260] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.162415 12559 consensus_queue.cc:260] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.162709 12560 ts_tablet_manager.cc:1428] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:45:57.162730 12546 heartbeater.cc:503] Master 127.20.105.126:34797 was elected leader, sending a full tablet report...
I20260808 00:45:57.162738 12559 ts_tablet_manager.cc:1428] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:45:57.162863 12357 heartbeater.cc:503] Master 127.20.105.126:34797 was elected leader, sending a full tablet report...
W20260808 00:45:57.211823 12358 tablet.cc:2369] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:45:57.300231 12154 tablet.cc:2369] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:45:57.363458 12547 tablet.cc:2369] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:45:57.369290 12563 raft_consensus.cc:482] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:45:57.369465 12563 raft_consensus.cc:504] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.369944 12563 leader_election.cc:288] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c090af398af14974ad700d27b72e4497 (127.20.105.66:38031), fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417)
I20260808 00:45:57.373265 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.370206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:38031 (local address 127.20.105.67:48203)
0808 00:45:57.370285 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:57.370301 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:57.370366 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:57.370625 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:57.370627 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:57.370629 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:57.370745 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.370748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.371493 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.371494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.372362 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.372364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.372968 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.372968 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.373055 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.373059 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:45:57.373126 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 00:45:57.373174 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 00:45:57.373265 12556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.370384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:48203 (local address 127.20.105.66:38031)
0808 00:45:57.370451 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.370453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.370469 (+ 16us) server_negotiation.cc:407] Connection header received
0808 00:45:57.370515 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.370517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.370530 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.370616 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:57.370841 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.371400 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.372460 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.372869 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.372961 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.372974 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:57.373132 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 00:45:57.373175 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:45:57.373744 12566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.370372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:37417 (local address 127.20.105.67:41877)
0808 00:45:57.370605 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:57.370618 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:45:57.370679 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:45:57.370969 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:45:57.370970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:45:57.370972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:45:57.371063 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.371065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.371819 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.371821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.372725 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:45:57.372728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.373463 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.373464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:45:57.373543 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.373548 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:45:57.373599 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:45:57.373649 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":164,"threads_started":1}
I20260808 00:45:57.373739 12568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.370491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:41877 (local address 127.20.105.65:37417)
0808 00:45:57.370651 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.370654 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.370665 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:45:57.370776 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.370777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.370828 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.370959 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:45:57.371148 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.371719 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.372829 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.373352 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.373447 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.373573 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:57.373603 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 00:45:57.373650 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260808 00:45:57.373730 12287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e11c945e7e7483b8876b1dd619b9ae3" candidate_uuid: "81b3baf2d1a24709a22033f547a76f95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c090af398af14974ad700d27b72e4497" is_pre_election: true
I20260808 00:45:57.373965 12287 raft_consensus.cc:2454] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81b3baf2d1a24709a22033f547a76f95 in term 0.
I20260808 00:45:57.374271 12379 leader_election.cc:302] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497; no voters:
I20260808 00:45:57.374419 12563 raft_consensus.cc:2781] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:45:57.374483 12563 raft_consensus.cc:482] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:45:57.374523 12563 raft_consensus.cc:3037] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:57.374428 12073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e11c945e7e7483b8876b1dd619b9ae3" candidate_uuid: "81b3baf2d1a24709a22033f547a76f95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3de2302cb84028b2b60596fe36e229" is_pre_election: true
I20260808 00:45:57.374651 12073 raft_consensus.cc:2454] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 81b3baf2d1a24709a22033f547a76f95 in term 0.
I20260808 00:45:57.375732 12563 raft_consensus.cc:504] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.375922 12563 leader_election.cc:288] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [CANDIDATE]: Term 1 election: Requested vote from peers c090af398af14974ad700d27b72e4497 (127.20.105.66:38031), fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417)
I20260808 00:45:57.376174 12287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e11c945e7e7483b8876b1dd619b9ae3" candidate_uuid: "81b3baf2d1a24709a22033f547a76f95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c090af398af14974ad700d27b72e4497"
I20260808 00:45:57.376164 12073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e11c945e7e7483b8876b1dd619b9ae3" candidate_uuid: "81b3baf2d1a24709a22033f547a76f95" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3de2302cb84028b2b60596fe36e229"
I20260808 00:45:57.376267 12287 raft_consensus.cc:3037] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:57.376291 12073 raft_consensus.cc:3037] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:45:57.377322 12287 raft_consensus.cc:2454] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81b3baf2d1a24709a22033f547a76f95 in term 1.
I20260808 00:45:57.377465 12073 raft_consensus.cc:2454] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 81b3baf2d1a24709a22033f547a76f95 in term 1.
I20260808 00:45:57.377563 12379 leader_election.cc:302] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497; no voters:
I20260808 00:45:57.377689 12563 raft_consensus.cc:2781] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:45:57.377912 12563 raft_consensus.cc:686] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 1 LEADER]: Becoming Leader. State: Replica: 81b3baf2d1a24709a22033f547a76f95, State: Running, Role: LEADER
I20260808 00:45:57.378048 12563 consensus_queue.cc:237] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:45:57.379244 11888 catalog_manager.cc:5495] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "81b3baf2d1a24709a22033f547a76f95" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } health_report { overall_health: UNKNOWN } } }
I20260808 00:45:57.385497 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.382798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58530 (local address 127.20.105.67:35827)
0808 00:45:57.382855 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:45:57.382857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:45:57.382868 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:45:57.382941 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:45:57.382942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:45:57.382961 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:45:57.383044 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:45:57.383220 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.383737 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.384566 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:45:57.384776 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:45:57.384867 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:45:57.385072 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:45:57.385201 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:45:57.385298 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:45:57.385339 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 00:45:57.385364 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:45:57.387895 12073 raft_consensus.cc:1261] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Refusing update from remote peer 81b3baf2d1a24709a22033f547a76f95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:57.387949 12287 raft_consensus.cc:1261] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 1 FOLLOWER]: Refusing update from remote peer 81b3baf2d1a24709a22033f547a76f95: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:45:57.388383 12569 consensus_queue.cc:1035] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:57.388567 12563 consensus_queue.cc:1035] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:45:57.391811 12575 mvcc.cc:205] Tried to move back new op lower bound from 7316070225457000448 to 7316070225421414400. Current Snapshot: MvccSnapshot[applied={T|T < 7316070225457000448}]
I20260808 00:45:57.392200 12576 mvcc.cc:205] Tried to move back new op lower bound from 7316070225457000448 to 7316070225421414400. Current Snapshot: MvccSnapshot[applied={T|T < 7316070225457000448}]
I20260808 00:45:57.394785 11887 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:57.394896 11887 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:45:57.395865 11887 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Sending DeleteTablet for 3 replicas of tablet 3e11c945e7e7483b8876b1dd619b9ae3
I20260808 00:45:57.396397 12055 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e11c945e7e7483b8876b1dd619b9ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:57 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37936
I20260808 00:45:57.396374 12443 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e11c945e7e7483b8876b1dd619b9ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:57 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:58518
I20260808 00:45:57.396482 12255 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e11c945e7e7483b8876b1dd619b9ae3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:45:57 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:34916
I20260808 00:45:57.396658 12588 tablet_replica.cc:330] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: stopping tablet replica
I20260808 00:45:57.396773 12588 raft_consensus.cc:2229] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:45:57.396852 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 11944
I20260808 00:45:57.396921 12588 raft_consensus.cc:2258] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:57.413012 12591 tablet_replica.cc:330] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: stopping tablet replica
I20260808 00:45:57.413275 12591 raft_consensus.cc:2229] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:45:57.413496 12591 raft_consensus.cc:2258] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:45:57.414139 12591 ts_tablet_manager.cc:1905] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:57.416203 12591 ts_tablet_manager.cc:1918] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:57.416354 12591 log.cc:1196] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/3e11c945e7e7483b8876b1dd619b9ae3
I20260808 00:45:57.416754 12591 ts_tablet_manager.cc:1939] T 3e11c945e7e7483b8876b1dd619b9ae3 P c090af398af14974ad700d27b72e4497: Deleting consensus metadata
I20260808 00:45:57.417393 11865 catalog_manager.cc:4841] TS c090af398af14974ad700d27b72e4497 (127.20.105.66:38031): tablet 3e11c945e7e7483b8876b1dd619b9ae3 (table test-table [id=d2948be73cda4adc886b2277ee27a80d]) successfully deleted
I20260808 00:45:57.422814 12588 ts_tablet_manager.cc:1905] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:45:57.425001 12588 ts_tablet_manager.cc:1918] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:45:57.425103 12588 log.cc:1196] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/3e11c945e7e7483b8876b1dd619b9ae3
I20260808 00:45:57.425463 12588 ts_tablet_manager.cc:1939] T 3e11c945e7e7483b8876b1dd619b9ae3 P 81b3baf2d1a24709a22033f547a76f95: Deleting consensus metadata
I20260808 00:45:57.426113 11869 catalog_manager.cc:4841] TS 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827): tablet 3e11c945e7e7483b8876b1dd619b9ae3 (table test-table [id=d2948be73cda4adc886b2277ee27a80d]) successfully deleted
W20260808 00:45:57.427997 11865 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:37417 (error 108)
I20260808 00:45:57.428669 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 12157
I20260808 00:45:57.428725 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:45:57.428549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:37417 (local address 127.0.0.1:37940)
0808 00:45:57.428616 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:45:57.428672 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 00:45:57.438663 11865 catalog_manager.cc:4584] TS fa3de2302cb84028b2b60596fe36e229 (127.20.105.65:37417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e11c945e7e7483b8876b1dd619b9ae3: Network error: Client connection negotiation failed: client connection to 127.20.105.65:37417: connect: Connection refused (error 111)
I20260808 00:45:57.452636 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 12360
I20260808 00:45:57.470932 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 11838
I20260808 00:46:17.490597 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34797
--webserver_interface=127.20.105.126
--webserver_port=42823
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:34797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:17.591549 12593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:17.591806 12593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:17.591831 12593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:17.593191 12593 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:46:17.593240 12593 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:46:17.593254 12593 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:46:17.593266 12593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:17.593281 12593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:17.593293 12593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:17.593305 12593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:17.593348 12593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:17.594969 12593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:34797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=42823
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:17.595206 12593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:17.595428 12593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:17.602003 12602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:17.601985 12603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:17.602138 12593 server_base.cc:1034] running on GCE node
W20260808 00:46:17.601990 12605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:17.602514 12593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:17.602843 12593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:17.604071 12593 hybrid_clock.cc:648] HybridClock initialized: now 1786149977604065 us; error 42 us; skew 500 ppm
Aug 08 00:46:17 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149977, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:17.612301 12593 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:46:17.612890 12593 webserver.cc:460] Webserver started at http://127.20.105.126:42823/ using document root <none> and password file <none>
I20260808 00:46:17.613198 12593 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:17.613253 12593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:17.614868 12593 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:46:17.615738 12614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:17.616055 12593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:17.616184 12593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "171543b6fa974aa0a9919280b50a9b9f"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "33087359d923cd7723077f15818d3cd6"
server_key_iv: "9b9e26ca745b3f4c592e13e8152f92fb"
server_key_version: "encryptionkey@0"
I20260808 00:46:17.616698 12593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:17.627025 12593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:17.630053 12593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:17.630213 12593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:17.635792 12593 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:34797
I20260808 00:46:17.635842 12698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:34797 every 8 connection(s)
I20260808 00:46:17.636436 12593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:17.638411 12699 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:46:17.639226 12699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Bootstrap starting.
I20260808 00:46:17.639564 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 12593
I20260808 00:46:17.639914 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:37417
--local_ip_for_outbound_sockets=127.20.105.65
--tserver_master_addrs=127.20.105.126:34797
--webserver_port=42329
--webserver_interface=127.20.105.65
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:46:17.650992 12699 log.cc:824] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:17.653944 12699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:46:17.654325 12699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Bootstrap complete.
I20260808 00:46:17.657147 12699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:46:17.657440 12699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 171543b6fa974aa0a9919280b50a9b9f, State: Initialized, Role: FOLLOWER
I20260808 00:46:17.657660 12699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:46:17.657752 12699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:17.657814 12699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:17.657873 12699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:46:17.659077 12699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:46:17.659214 12699 leader_election.cc:302] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 171543b6fa974aa0a9919280b50a9b9f; no voters:
I20260808 00:46:17.659453 12699 leader_election.cc:288] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 00:46:17.659509 12704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:46:17.659673 12704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [term 3 LEADER]: Becoming Leader. State: Replica: 171543b6fa974aa0a9919280b50a9b9f, State: Running, Role: LEADER
I20260808 00:46:17.659775 12704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } }
I20260808 00:46:17.659827 12699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:17.660112 12705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "171543b6fa974aa0a9919280b50a9b9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } } }
I20260808 00:46:17.660208 12705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:17.660531 12707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 171543b6fa974aa0a9919280b50a9b9f. Latest consensus state: current_term: 3 leader_uuid: "171543b6fa974aa0a9919280b50a9b9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "171543b6fa974aa0a9919280b50a9b9f" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34797 } } }
I20260808 00:46:17.660642 12707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:17.660785 12710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:17.661738 12710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:17.661851 12710 catalog_manager.cc:1259] Loaded cluster ID: c38bf853828d4318b9a4f6ed8ef6bd68
I20260808 00:46:17.661880 12710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:17.663273 12710 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:17.663499 12710 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Loaded TSK: 0
I20260808 00:46:17.663553 12710 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 171543b6fa974aa0a9919280b50a9b9f: Loaded TSK: 1
F20260808 00:46:17.663817 12710 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f405a0d49ad google::LogMessage::Fail() at ??:0
@ 0x7f405a0d788a google::LogMessage::SendToLog() at ??:0
@ 0x7f405a0d44de google::LogMessage::Flush() at ??:0
@ 0x7f405a0d5a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f405c841104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f405c83ed41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f406202e1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f406202e48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f406202e594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f4061ff4c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f4061ffead0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f405bfa20d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f405bf97954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f405cdfb6db start_thread at ??:0
@ 0x7f4058bd261f clone at ??:0
W20260808 00:46:17.781512 12701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:17.781854 12701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:17.781898 12701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:17.783838 12701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:17.783926 12701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:17.783975 12701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:46:17.786615 12701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:37417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=42329
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:17.787016 12701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:17.787400 12701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:17.794219 12757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:17.794234 12755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:17.794256 12753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:17.795478 12701 server_base.cc:1034] running on GCE node
I20260808 00:46:17.795763 12701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:17.796147 12701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:17.797348 12701 hybrid_clock.cc:648] HybridClock initialized: now 1786149977797335 us; error 37 us; skew 500 ppm
Aug 08 00:46:17 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149977, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:17.800647 12701 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:46:17.801222 12701 webserver.cc:460] Webserver started at http://127.20.105.65:42329/ using document root <none> and password file <none>
I20260808 00:46:17.801537 12701 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:17.801618 12701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:17.803300 12701 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:17.804167 12767 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:17.804360 12701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:46:17.804497 12701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "fa3de2302cb84028b2b60596fe36e229"
format_stamp: "Formatted at 2026-08-08 00:45:35 on dist-test-slave-kvfs"
server_key: "e818fb19683185a8f0c45839a39719e0"
server_key_iv: "071331a0ba3aa3a2b7b8faeff5a142f6"
server_key_version: "encryptionkey@0"
I20260808 00:46:17.804952 12701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:17.830389 12701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:17.833308 12701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:17.833516 12701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:17.833828 12701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:17.834419 12776 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:46:17.836854 12701 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:46:17.836939 12701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260808 00:46:17.837000 12701 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:46:17.837846 12701 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:46:17.837908 12701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:46:17.837998 12776 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Bootstrap starting.
I20260808 00:46:17.847348 12701 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:37417
I20260808 00:46:17.847390 12932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:37417 every 8 connection(s)
I20260808 00:46:17.847959 12701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:46:17.849258 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 12701
I20260808 00:46:17.849596 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:38031
--local_ip_for_outbound_sockets=127.20.105.66
--tserver_master_addrs=127.20.105.126:34797
--webserver_port=45333
--webserver_interface=127.20.105.66
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:46:17.851949 12776 log.cc:824] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:17.856043 12776 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:46:17.856408 12776 tablet_bootstrap.cc:492] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Bootstrap complete.
I20260808 00:46:17.857762 12776 ts_tablet_manager.cc:1397] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.004s
I20260808 00:46:17.859450 12776 raft_consensus.cc:348] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:46:17.859833 12776 raft_consensus.cc:729] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3de2302cb84028b2b60596fe36e229, State: Initialized, Role: FOLLOWER
I20260808 00:46:17.860085 12776 consensus_queue.cc:260] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:46:17.860478 12776 ts_tablet_manager.cc:1428] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
W20260808 00:46:17.926604 12936 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:46:17.848912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:57293)
0808 00:46:17.853701 (+ 4789us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:17.853759 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:17.854705 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:17.926298 (+ 71593us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":4730,"thread_start_us":128,"threads_started":1}
W20260808 00:46:17.927011 12933 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:34797 (0 consecutive failures): Network error: Failed to ping master at 127.20.105.126:34797: Client connection negotiation failed: client connection to 127.20.105.126:34797: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:46:17.927551 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:17.927356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:58635)
0808 00:46:17.927462 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:17.927521 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 00:46:17.928066 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:17.927910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:46325)
0808 00:46:17.927965 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:17.928002 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:46:17.928221 12933 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:46:17.958907 12937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:17.959216 12937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:17.959262 12937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:17.960631 12937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:17.960700 12937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:17.960754 12937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:46:17.962337 12937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:38031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=45333
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:17.962641 12937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:17.962917 12937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:17.968930 12951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:17.968906 12947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:17.969082 12946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:17.969321 12937 server_base.cc:1034] running on GCE node
I20260808 00:46:17.969594 12937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:17.969901 12937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:17.971078 12937 hybrid_clock.cc:648] HybridClock initialized: now 1786149977971055 us; error 43 us; skew 500 ppm
Aug 08 00:46:17 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149977, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:17.974422 12937 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:46:17.974978 12937 webserver.cc:460] Webserver started at http://127.20.105.66:45333/ using document root <none> and password file <none>
I20260808 00:46:17.975244 12937 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:17.975320 12937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:17.977002 12937 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:17.977769 12963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:17.977996 12937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:17.978134 12937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "c090af398af14974ad700d27b72e4497"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "711fdbb9bccb5bb25432f14134f181a1"
server_key_iv: "b1bd8658f75ef579900c85b7e99c3f1c"
server_key_version: "encryptionkey@0"
I20260808 00:46:17.978585 12937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:18.013840 12937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:18.016476 12937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:18.016671 12937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:18.017016 12937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:18.017432 12937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:18.017491 12937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:18.017537 12937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:18.017567 12937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:18.025231 12937 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:38031
I20260808 00:46:18.025286 13130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:38031 every 8 connection(s)
I20260808 00:46:18.025801 12937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:46:18.027289 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:18.026766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:45499)
0808 00:46:18.026982 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:18.027084 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
W20260808 00:46:18.027822 13131 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:34797 (0 consecutive failures): Network error: Failed to ping master at 127.20.105.126:34797: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
I20260808 00:46:18.028230 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:18.028118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:36451)
0808 00:46:18.028169 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:18.028208 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 00:46:18.028573 13134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:18.028466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:48559)
0808 00:46:18.028524 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:18.028550 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 00:46:18.028676 13131 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:46:18.030223 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 12937
I20260808 00:46:18.030490 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:35827
--local_ip_for_outbound_sockets=127.20.105.67
--tserver_master_addrs=127.20.105.126:34797
--webserver_port=39441
--webserver_interface=127.20.105.67
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:18.131333 13135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:18.131642 13135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:18.131683 13135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:18.132998 13135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:18.133080 13135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:18.133127 13135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:46:18.134737 13135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:35827
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=39441
--tserver_master_addrs=127.20.105.126:34797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:18.135072 13135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:18.135358 13135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:18.141542 13146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:18.141620 13135 server_base.cc:1034] running on GCE node
W20260808 00:46:18.141434 13143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:18.141601 13142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:18.142004 13135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:18.142292 13135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:18.143464 13135 hybrid_clock.cc:648] HybridClock initialized: now 1786149978143451 us; error 53 us; skew 500 ppm
Aug 08 00:46:18 dist-test-slave-kvfs krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149978, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:18.146466 13135 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:46:18.147073 13135 webserver.cc:460] Webserver started at http://127.20.105.67:39441/ using document root <none> and password file <none>
I20260808 00:46:18.147347 13135 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:18.147423 13135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:18.149129 13135 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:18.149853 13155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:18.150053 13135 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:18.150159 13135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "81b3baf2d1a24709a22033f547a76f95"
format_stamp: "Formatted at 2026-08-08 00:45:36 on dist-test-slave-kvfs"
server_key: "09ffb72d699fb3cf2eb63abaca9f81d9"
server_key_iv: "0d45b46595857fac00ec1479f914ee7b"
server_key_version: "encryptionkey@0"
I20260808 00:46:18.150573 13135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:18.161492 13135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:18.164145 13135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:18.164275 13135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:18.164505 13135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:18.164810 13135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:18.164840 13135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:18.164865 13135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:18.164878 13135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:18.172293 13135 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:35827
I20260808 00:46:18.172345 13330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:35827 every 8 connection(s)
I20260808 00:46:18.172762 13135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:46:18.174065 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:18.173562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:34167)
0808 00:46:18.173788 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:18.173878 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
W20260808 00:46:18.174583 13331 heartbeater.cc:644] Failed to heartbeat to 127.20.105.126:34797 (0 consecutive failures): Network error: Failed to ping master at 127.20.105.126:34797: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
I20260808 00:46:18.174923 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:18.174806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:50051)
0808 00:46:18.174863 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:18.174891 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 00:46:18.175207 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:18.175119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:33037)
0808 00:46:18.175164 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:18.175187 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 00:46:18.175323 13331 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:46:18.178855 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 13135
I20260808 00:46:18.929168 13336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:18.928820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:51437)
0808 00:46:18.929017 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:18.929075 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":87,"threads_started":1}
I20260808 00:46:19.029338 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:19.029060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:49555)
0808 00:46:19.029230 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:19.029273 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260808 00:46:19.175904 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:19.175666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:39695)
0808 00:46:19.175806 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:19.175841 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260808 00:46:19.717842 13340 raft_consensus.cc:482] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:19.718091 13340 raft_consensus.cc:504] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:46:19.718580 13340 leader_election.cc:288] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c090af398af14974ad700d27b72e4497 (127.20.105.66:38031), 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827)
Aug 08 00:46:19 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149977, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.67@KRBTEST.COM
Aug 08 00:46:19 dist-test-slave-kvfs krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149977, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.66@KRBTEST.COM
I20260808 00:46:19.733379 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:19.719061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:45709 (local address 127.20.105.67:35827)
0808 00:46:19.719274 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:19.719282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:19.719302 (+ 20us) server_negotiation.cc:407] Connection header received
0808 00:46:19.720071 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:19.720074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:19.720139 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:19.720328 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:19.721589 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:19.723120 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:19.724871 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:19.725577 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:19.725717 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:19.729256 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:19.729261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:19.729261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:19.729276 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:19.731385 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:19.731889 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:19.731894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:19.731894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:19.732009 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:19.732342 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:19.732343 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:19.732343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:19.732649 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:19.732752 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:19.733092 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 00:46:19.733236 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
I20260808 00:46:19.733366 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:19.718875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:35827 (local address 127.20.105.65:45709)
0808 00:46:19.719034 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:19.719048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:19.719133 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:19.720419 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:19.720423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:19.721042 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:19.721404 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:19.721414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:19.723265 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:19.723267 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:19.724758 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:19.724762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:19.725710 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:19.725711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:19.725897 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:19.726688 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:19.726701 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:19.729093 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:19.731493 (+ 2400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:19.731504 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:19.731507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:19.731774 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:19.732131 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:19.732132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:19.732132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:19.732258 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:19.732725 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:19.732733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:19.732886 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 00:46:19.733021 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 00:46:19.733251 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":92,"thread_start_us":78,"threads_started":1}
I20260808 00:46:19.733582 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:19.718844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:38031 (local address 127.20.105.65:40195)
0808 00:46:19.719033 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:19.719048 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:19.719133 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:19.720419 (+ 1286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:19.720423 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:19.721042 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:19.721404 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:19.721414 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:19.723285 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:19.723286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:19.724758 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:19.724762 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:19.725709 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:19.725710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:19.725897 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:19.726688 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:19.726701 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:19.729457 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:19.731841 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:19.731843 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:19.731844 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:19.732122 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:19.732451 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:19.732452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:19.732453 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:19.732531 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:19.732973 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:19.732974 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:19.733029 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 00:46:19.733121 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:46:19.733360 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":130,"thread_start_us":113,"threads_started":1}
I20260808 00:46:19.733932 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:19.719015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:40195 (local address 127.20.105.66:38031)
0808 00:46:19.719240 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:19.719258 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:19.719293 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:46:19.720057 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:19.720062 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:19.720137 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:19.720328 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:19.721587 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:19.723121 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:19.724871 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:19.725577 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:19.725717 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:19.729608 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:19.729613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:19.729613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:19.729627 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:19.731717 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:19.732237 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:19.732241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:19.732241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:19.732351 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:19.732611 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:19.732612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:19.732612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:19.732880 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:19.732977 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:19.733205 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 00:46:19.733377 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
W20260808 00:46:19.734651 12772 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
W20260808 00:46:19.734890 12768 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c090af398af14974ad700d27b72e4497 (127.20.105.66:38031): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
I20260808 00:46:19.734987 12768 leader_election.cc:302] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3de2302cb84028b2b60596fe36e229; no voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497
I20260808 00:46:19.735212 13340 raft_consensus.cc:2726] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:46:19.930332 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:19.930102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:60703)
0808 00:46:19.930202 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:19.930262 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 00:46:20.030287 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:20.029923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:34365)
0808 00:46:20.030115 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:20.030195 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":109,"threads_started":1}
I20260808 00:46:20.176967 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:20.176572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:33623)
0808 00:46:20.176800 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:20.176889 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":84,"threads_started":1}
I20260808 00:46:20.931243 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:20.930948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:36825)
0808 00:46:20.931123 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:20.931171 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260808 00:46:21.031180 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:21.030870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:51647)
0808 00:46:21.031046 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:21.031110 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":118,"threads_started":1}
I20260808 00:46:21.177875 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:21.177539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:47013)
0808 00:46:21.177748 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:21.177798 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":96,"threads_started":1}
I20260808 00:46:21.240746 13356 raft_consensus.cc:482] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:21.240918 13356 raft_consensus.cc:504] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:46:21.241487 13356 leader_election.cc:288] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c090af398af14974ad700d27b72e4497 (127.20.105.66:38031), 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827)
W20260808 00:46:21.241981 12768 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c090af398af14974ad700d27b72e4497 (127.20.105.66:38031): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
W20260808 00:46:21.242147 12772 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
I20260808 00:46:21.242228 12772 leader_election.cc:302] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3de2302cb84028b2b60596fe36e229; no voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497
I20260808 00:46:21.242312 13356 raft_consensus.cc:2726] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:46:21.932365 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:21.931914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:45849)
0808 00:46:21.932214 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:21.932272 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":156,"threads_started":1}
I20260808 00:46:22.032294 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:22.031921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:41581)
0808 00:46:22.032172 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:22.032225 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":166,"threads_started":1}
I20260808 00:46:22.178953 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:22.178641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:41511)
0808 00:46:22.178837 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:22.178890 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260808 00:46:22.819190 13362 raft_consensus.cc:482] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:22.819339 13362 raft_consensus.cc:504] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:46:22.819602 13362 leader_election.cc:288] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c090af398af14974ad700d27b72e4497 (127.20.105.66:38031), 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827)
W20260808 00:46:22.820215 12768 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c090af398af14974ad700d27b72e4497 (127.20.105.66:38031): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
W20260808 00:46:22.820331 12772 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
I20260808 00:46:22.820446 12768 leader_election.cc:302] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3de2302cb84028b2b60596fe36e229; no voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497
I20260808 00:46:22.820576 13362 raft_consensus.cc:2726] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:46:22.933466 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:22.933150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:33031)
0808 00:46:22.933348 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:22.933394 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":92,"threads_started":1}
I20260808 00:46:23.033215 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:23.032899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:40017)
0808 00:46:23.033096 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:23.033149 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":91,"threads_started":1}
I20260808 00:46:23.180110 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:23.179665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:34025)
0808 00:46:23.179896 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:23.179958 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":125,"threads_started":1}
I20260808 00:46:23.934535 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:23.934190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:60515)
0808 00:46:23.934398 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:23.934449 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
I20260808 00:46:24.034198 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:24.033849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:52171)
0808 00:46:24.034054 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:24.034125 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":95,"threads_started":1}
I20260808 00:46:24.181138 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:24.180753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:57451)
0808 00:46:24.181020 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:24.181065 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":192,"threads_started":1}
I20260808 00:46:24.936074 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:24.935529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:34863)
0808 00:46:24.935874 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:24.935929 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":175,"threads_started":1}
I20260808 00:46:25.035301 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:25.034958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:52367)
0808 00:46:25.035174 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:25.035228 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":103,"threads_started":1}
I20260808 00:46:25.182216 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:25.181878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:51105)
0808 00:46:25.182087 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:25.182144 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":123,"threads_started":1}
I20260808 00:46:25.841813 13377 raft_consensus.cc:482] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:25.841969 13377 raft_consensus.cc:504] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:46:25.842262 13377 leader_election.cc:288] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c090af398af14974ad700d27b72e4497 (127.20.105.66:38031), 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827)
W20260808 00:46:25.842744 12768 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c090af398af14974ad700d27b72e4497 (127.20.105.66:38031): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
W20260808 00:46:25.842852 12772 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
I20260808 00:46:25.842952 12772 leader_election.cc:302] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3de2302cb84028b2b60596fe36e229; no voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497
I20260808 00:46:25.843067 13377 raft_consensus.cc:2726] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:46:25.937249 13378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:25.936886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:46329)
0808 00:46:25.937111 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:25.937172 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":98,"threads_started":1}
I20260808 00:46:26.036480 13380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:26.036143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:58347)
0808 00:46:26.036365 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:26.036412 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":111,"threads_started":1}
I20260808 00:46:26.183306 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:26.182938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:54913)
0808 00:46:26.183176 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:26.183234 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":120,"threads_started":1}
I20260808 00:46:26.938391 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:26.938064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:48959)
0808 00:46:26.938261 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:26.938319 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":95,"threads_started":1}
I20260808 00:46:27.037498 13383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:27.037143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:45865)
0808 00:46:27.037357 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:27.037420 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":97,"threads_started":1}
I20260808 00:46:27.184420 13384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:27.183966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:49451)
0808 00:46:27.184255 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:27.184330 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":188,"threads_started":1}
I20260808 00:46:27.939680 13385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:27.939287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:38351)
0808 00:46:27.939543 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:27.939605 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":114,"threads_started":1}
I20260808 00:46:28.038707 13386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:28.038301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:53465)
0808 00:46:28.038540 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:28.038614 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":92,"threads_started":1}
I20260808 00:46:28.185437 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:28.185107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:42965)
0808 00:46:28.185301 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:28.185357 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":97,"threads_started":1}
I20260808 00:46:28.940946 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:28.940603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:51105)
0808 00:46:28.940807 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:28.940867 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":95,"threads_started":1}
I20260808 00:46:29.039738 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:29.039412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:39819)
0808 00:46:29.039606 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:29.039665 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":101,"threads_started":1}
I20260808 00:46:29.186501 13391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:29.186198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:48833)
0808 00:46:29.186384 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:29.186435 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":99,"threads_started":1}
I20260808 00:46:29.942042 13392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:29.941668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:50507)
0808 00:46:29.941894 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:29.941952 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":123,"threads_started":1}
I20260808 00:46:30.040895 13394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:30.040517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:55265)
0808 00:46:30.040770 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:30.040827 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
I20260808 00:46:30.187547 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:30.187207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:35431)
0808 00:46:30.187421 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:30.187474 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260808 00:46:30.943105 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:30.942796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:42395)
0808 00:46:30.942990 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:30.943032 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":91,"threads_started":1}
I20260808 00:46:31.041913 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:31.041578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:36277)
0808 00:46:31.041776 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:31.041832 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260808 00:46:31.188665 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:31.188318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:42873)
0808 00:46:31.188530 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:31.188592 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":110,"threads_started":1}
I20260808 00:46:31.944309 13401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:31.943910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:45715)
0808 00:46:31.944146 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:31.944210 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":151,"threads_started":1}
I20260808 00:46:32.042968 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:32.042628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:45249)
0808 00:46:32.042840 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:32.042890 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":94,"threads_started":1}
I20260808 00:46:32.189819 13404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:32.189482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:57105)
0808 00:46:32.189682 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:32.189742 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
I20260808 00:46:32.791955 13406 raft_consensus.cc:482] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:32.792198 13406 raft_consensus.cc:504] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c090af398af14974ad700d27b72e4497" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 38031 } } peers { permanent_uuid: "81b3baf2d1a24709a22033f547a76f95" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 35827 } } peers { permanent_uuid: "fa3de2302cb84028b2b60596fe36e229" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 37417 } }
I20260808 00:46:32.792439 13406 leader_election.cc:288] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c090af398af14974ad700d27b72e4497 (127.20.105.66:38031), 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827)
W20260808 00:46:32.793100 12772 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 81b3baf2d1a24709a22033f547a76f95 (127.20.105.67:35827): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
W20260808 00:46:32.793200 12768 leader_election.cc:341] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c090af398af14974ad700d27b72e4497 (127.20.105.66:38031): Not found: Tablet not found: 3e11c945e7e7483b8876b1dd619b9ae3
I20260808 00:46:32.793232 12772 leader_election.cc:302] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fa3de2302cb84028b2b60596fe36e229; no voters: 81b3baf2d1a24709a22033f547a76f95, c090af398af14974ad700d27b72e4497
I20260808 00:46:32.793318 13406 raft_consensus.cc:2726] T 3e11c945e7e7483b8876b1dd619b9ae3 P fa3de2302cb84028b2b60596fe36e229 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:46:32.945391 13408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:32.945070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.65:60763)
0808 00:46:32.945278 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:32.945318 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":129,"threads_started":1}
I20260808 00:46:33.044106 13409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:33.043749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.66:50303)
0808 00:46:33.043934 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:33.043976 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":107,"threads_started":1}
I20260808 00:46:33.181540 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 12701
I20260808 00:46:33.193687 13410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:33.193375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34797 (local address 127.20.105.67:42397)
0808 00:46:33.193559 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:33.193614 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.126:34797: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260808 00:46:33.200444 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 12937
I20260808 00:46:33.215503 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 13135
2026-08-08T00:46:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59245 ms)
[----------] 30 tests from SecurityITest (263518 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:33 dist-test-slave-kvfs krb5kdc[13416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:33 dist-test-slave-kvfs krb5kdc[13416](info): set up 2 sockets
Aug 08 00:46:33 dist-test-slave-kvfs krb5kdc[13416](info): commencing operation
krb5kdc: starting...
W20260808 00:46:34.504199 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.005s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:34 dist-test-slave-kvfs krb5kdc[13416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:46:34.656140 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40081
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:39729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:40081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:34.754374 13434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:34.754648 13434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:34.754676 13434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:34.755973 13434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:34.756047 13434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:34.756103 13434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:34.756135 13434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:34.756151 13434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:34.757802 13434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:39729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:40081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:34.758064 13434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:34.758299 13434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:34.764209 13440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:34.764220 13441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:34.764220 13445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:34.764509 13434 server_base.cc:1034] running on GCE node
I20260808 00:46:34.764809 13434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:34.765159 13434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:34.766402 13434 hybrid_clock.cc:648] HybridClock initialized: now 1786149994766378 us; error 41 us; skew 500 ppm
Aug 08 00:46:34 dist-test-slave-kvfs krb5kdc[13416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149994, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:34.769416 13434 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:46:34.769990 13434 webserver.cc:460] Webserver started at http://127.20.105.126:43805/ using document root <none> and password file <none>
I20260808 00:46:34.770285 13434 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:34.770361 13434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:34.770533 13434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:34.771797 13434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "52028e5ecb014ab9893e0545eefb162d"
format_stamp: "Formatted at 2026-08-08 00:46:34 on dist-test-slave-kvfs"
server_key: "ba6d5952a41fa5cd24fe1117bd424d3e"
server_key_iv: "a7454710e1813f4432df47c0eb575aee"
server_key_version: "encryptionkey@0"
I20260808 00:46:34.772281 13434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "52028e5ecb014ab9893e0545eefb162d"
format_stamp: "Formatted at 2026-08-08 00:46:34 on dist-test-slave-kvfs"
server_key: "ba6d5952a41fa5cd24fe1117bd424d3e"
server_key_iv: "a7454710e1813f4432df47c0eb575aee"
server_key_version: "encryptionkey@0"
I20260808 00:46:34.773856 13434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:34.774744 13455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:34.774968 13434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:34.775090 13434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "52028e5ecb014ab9893e0545eefb162d"
format_stamp: "Formatted at 2026-08-08 00:46:34 on dist-test-slave-kvfs"
server_key: "ba6d5952a41fa5cd24fe1117bd424d3e"
server_key_iv: "a7454710e1813f4432df47c0eb575aee"
server_key_version: "encryptionkey@0"
I20260808 00:46:34.775174 13434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:34.810086 13434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:34.812732 13434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:34.812929 13434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:34.818370 13434 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:40081
I20260808 00:46:34.818414 13534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:40081 every 8 connection(s)
I20260808 00:46:34.818871 13434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:34.819986 13538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:34.822842 13538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d: Bootstrap starting.
I20260808 00:46:34.823549 13538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:34.823856 13538 log.cc:824] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:34.824704 13538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d: No bootstrap required, opened a new log
I20260808 00:46:34.825009 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 13434
I20260808 00:46:34.825161 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:34.825371 20901 external_mini_cluster.cc:1428] Setting key 904773788e358fe70ed43b3d97686714
I20260808 00:46:34.827368 13538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52028e5ecb014ab9893e0545eefb162d" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40081 } }
I20260808 00:46:34.827613 13538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:34.827661 13538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52028e5ecb014ab9893e0545eefb162d, State: Initialized, Role: FOLLOWER
I20260808 00:46:34.827857 13538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52028e5ecb014ab9893e0545eefb162d" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40081 } }
I20260808 00:46:34.827961 13538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:34.827989 13538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:34.828075 13538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:34.829502 13538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52028e5ecb014ab9893e0545eefb162d" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40081 } }
I20260808 00:46:34.829648 13538 leader_election.cc:302] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52028e5ecb014ab9893e0545eefb162d; no voters:
I20260808 00:46:34.829852 13538 leader_election.cc:288] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:34.829964 13547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:34.830142 13538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:34.830181 13547 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [term 1 LEADER]: Becoming Leader. State: Replica: 52028e5ecb014ab9893e0545eefb162d, State: Running, Role: LEADER
I20260808 00:46:34.830354 13547 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52028e5ecb014ab9893e0545eefb162d" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40081 } }
I20260808 00:46:34.830910 13550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52028e5ecb014ab9893e0545eefb162d. Latest consensus state: current_term: 1 leader_uuid: "52028e5ecb014ab9893e0545eefb162d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52028e5ecb014ab9893e0545eefb162d" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40081 } } }
I20260808 00:46:34.831056 13550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:34.830899 13549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52028e5ecb014ab9893e0545eefb162d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52028e5ecb014ab9893e0545eefb162d" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40081 } } }
I20260808 00:46:34.831204 13549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:34.832104 13562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:34.832748 13562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:34.834910 13562 catalog_manager.cc:1347] Generated new cluster ID: 40f25f9da226470397a403f43b7d4a00
I20260808 00:46:34.834968 13562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:46:34 dist-test-slave-kvfs krb5kdc[13416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:34.840353 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:34.826113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45954 (local address 127.20.105.126:40081)
0808 00:46:34.826360 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:34.826372 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:34.826410 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:46:34.827310 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:34.827322 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:34.827401 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:34.827606 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:34.828593 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:34.830190 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:34.831528 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:34.832406 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:34.832500 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:34.835950 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:34.835955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:34.835955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:34.835970 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:34.838227 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:34.838667 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:34.838671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:34.838672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:34.838827 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:34.839063 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:34.839064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:34.839065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:34.839457 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:34.839670 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:34.839756 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 00:46:34.840097 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":125,"threads_started":1}
I20260808 00:46:34.842433 13562 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:34.843222 13562 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:34.853387 13562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d: Generated new TSK 0
I20260808 00:46:34.853703 13562 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:46:34.891891 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40081
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:39729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:34.992748 13579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:34.993049 13579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:34.993093 13579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:34.994374 13579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:34.994450 13579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:34.994498 13579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:46:34.996270 13579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:39729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:34.996595 13579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:34.996904 13579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:35.003298 13588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:35.003283 13594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:35.003279 13587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:35.003660 13579 server_base.cc:1034] running on GCE node
I20260808 00:46:35.003868 13579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:35.004211 13579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:35.005391 13579 hybrid_clock.cc:648] HybridClock initialized: now 1786149995005367 us; error 42 us; skew 500 ppm
Aug 08 00:46:35 dist-test-slave-kvfs krb5kdc[13416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:35.008724 13579 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:46:35.009294 13579 webserver.cc:460] Webserver started at http://127.20.105.65:43047/ using document root <none> and password file <none>
I20260808 00:46:35.009596 13579 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:35.009675 13579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:35.009840 13579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:35.011171 13579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "f31e33869e394a75ad2b1798466aa277"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "cf10e9c098c6ce38d24a5645ac2f3939"
server_key_iv: "66e9767b069e550066e2331d5223014d"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.011620 13579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "f31e33869e394a75ad2b1798466aa277"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "cf10e9c098c6ce38d24a5645ac2f3939"
server_key_iv: "66e9767b069e550066e2331d5223014d"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.013325 13579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 00:46:35.014400 13604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.014642 13579 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:46:35.014768 13579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "f31e33869e394a75ad2b1798466aa277"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "cf10e9c098c6ce38d24a5645ac2f3939"
server_key_iv: "66e9767b069e550066e2331d5223014d"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.014859 13579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:35.035704 13579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:35.038213 13579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:35.038410 13579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:35.038748 13579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:35.039162 13579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:35.039222 13579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.039271 13579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:35.039299 13579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.046681 13579 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:43921
I20260808 00:46:35.046741 13763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:43921 every 8 connection(s)
I20260808 00:46:35.047168 13579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:46:35.050863 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 13579
I20260808 00:46:35.051023 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:46:35.051247 20901 external_mini_cluster.cc:1428] Setting key e53ac3eab2ece412f8607c6f86051313
Aug 08 00:46:35 dist-test-slave-kvfs krb5kdc[13416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:35.058728 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.048191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:45495 (local address 127.20.105.126:40081)
0808 00:46:35.048248 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.048250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.048543 (+ 293us) server_negotiation.cc:407] Connection header received
0808 00:46:35.049221 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.049223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.049244 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.049313 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:35.050468 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.051099 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.052456 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.052696 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.052791 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.055447 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:35.055448 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:35.055449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:35.055462 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:35.057096 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:35.057545 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:35.057547 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:35.057547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:35.057616 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:35.057882 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:35.057883 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:35.057884 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:35.058095 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:35.058183 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.058490 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.058617 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:46:35.058938 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.048180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40081 (local address 127.20.105.65:45495)
0808 00:46:35.048385 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.048430 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:35.049079 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:35.049412 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:35.049416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:35.050105 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:35.050349 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.050355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.051201 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.051202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.052350 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.052354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.052817 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.052819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.052953 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.053608 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:35.053619 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:35.055339 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:35.057195 (+ 1856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:35.057203 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:35.057206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:35.057444 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:35.057686 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:35.057687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:35.057687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:35.057810 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:35.058183 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:35.058192 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:35.058349 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 00:46:35.058467 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 00:46:35.058652 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":100,"threads_started":1}
I20260808 00:46:35.059696 13764 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40081
I20260808 00:46:35.059818 13764 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:35.060091 13764 heartbeater.cc:511] Master 127.20.105.126:40081 requested a full tablet report, sending...
I20260808 00:46:35.060873 13482 ts_manager.cc:194] Registered new tserver with Master: f31e33869e394a75ad2b1798466aa277 (127.20.105.65:43921)
I20260808 00:46:35.061926 13482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:45495
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:46:35.084264 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40081
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:39729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:35.184149 13770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:35.184423 13770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:35.184453 13770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:35.185699 13770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:35.185739 13770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:35.185767 13770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:46:35.187282 13770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:39729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:35.187512 13770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:35.187757 13770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:35.193979 13782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:35.194060 13770 server_base.cc:1034] running on GCE node
W20260808 00:46:35.193986 13777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:35.193992 13776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:35.194423 13770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:35.194672 13770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:35.195822 13770 hybrid_clock.cc:648] HybridClock initialized: now 1786149995195810 us; error 37 us; skew 500 ppm
Aug 08 00:46:35 dist-test-slave-kvfs krb5kdc[13416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:35.199121 13770 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:46:35.199657 13770 webserver.cc:460] Webserver started at http://127.20.105.66:34911/ using document root <none> and password file <none>
I20260808 00:46:35.199926 13770 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:35.199997 13770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:35.200179 13770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:35.201431 13770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "3077a24f8b1a4eecad667c4b74dc7ae8"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "536536b1b7110ce5f7056e885f37a31f"
server_key_iv: "c67024e04a91a65b5a8f0e278510b495"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.201869 13770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "3077a24f8b1a4eecad667c4b74dc7ae8"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "536536b1b7110ce5f7056e885f37a31f"
server_key_iv: "c67024e04a91a65b5a8f0e278510b495"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.203428 13770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:46:35.204316 13791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.204517 13770 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:35.204632 13770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "3077a24f8b1a4eecad667c4b74dc7ae8"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "536536b1b7110ce5f7056e885f37a31f"
server_key_iv: "c67024e04a91a65b5a8f0e278510b495"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.204711 13770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:35.216534 13770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:35.219269 13770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:35.219453 13770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:35.219756 13770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:35.220254 13770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:35.220310 13770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.220350 13770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:35.220376 13770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.227710 13770 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:36623
I20260808 00:46:35.227754 13965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:36623 every 8 connection(s)
I20260808 00:46:35.228236 13770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:46:35.232616 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 13770
I20260808 00:46:35.232746 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:46:35.232997 20901 external_mini_cluster.cc:1428] Setting key 794f1c9b9d3b26cfdd2f44a2751d8935
Aug 08 00:46:35 dist-test-slave-kvfs krb5kdc[13416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:35.240811 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.229236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:57917 (local address 127.20.105.126:40081)
0808 00:46:35.229316 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.229318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.229621 (+ 303us) server_negotiation.cc:407] Connection header received
0808 00:46:35.230443 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.230450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.230468 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.230551 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:35.231940 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.232704 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.234316 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.234564 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.234651 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.237563 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:35.237567 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:35.237567 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:35.237583 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:35.239122 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:35.239605 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:35.239606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:35.239607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:35.239678 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:35.239958 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:35.239959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:35.239959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:35.240164 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:35.240248 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.240580 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.240696 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:46:35.241073 13972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.229195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40081 (local address 127.20.105.66:57917)
0808 00:46:35.229424 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.229480 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:35.230233 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:35.230636 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:35.230641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:35.231534 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:35.231806 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.231812 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.232890 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.232892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.234213 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.234217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.234678 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.234679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.234834 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.235559 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:35.235570 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:35.237438 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:35.239234 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:35.239244 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:35.239247 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:35.239521 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:35.239751 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:35.239751 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:35.239752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:35.239886 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:35.240238 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:35.240245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:35.240424 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 00:46:35.240556 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 00:46:35.240763 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":116,"threads_started":1}
I20260808 00:46:35.241909 13966 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40081
I20260808 00:46:35.242098 13966 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:35.242367 13966 heartbeater.cc:511] Master 127.20.105.126:40081 requested a full tablet report, sending...
I20260808 00:46:35.242975 13482 ts_manager.cc:194] Registered new tserver with Master: 3077a24f8b1a4eecad667c4b74dc7ae8 (127.20.105.66:36623)
I20260808 00:46:35.243592 13482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:57917
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:46:35.272522 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40081
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:39729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:35.375108 13976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:35.375347 13976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:35.375375 13976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:35.376592 13976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:35.376634 13976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:35.376662 13976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:46:35.378134 13976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:39729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:40081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:35.378456 13976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:35.378728 13976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:35.384809 13983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:35.384836 13989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:35.384828 13985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:35.385041 13976 server_base.cc:1034] running on GCE node
I20260808 00:46:35.385380 13976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:35.385674 13976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:35.386859 13976 hybrid_clock.cc:648] HybridClock initialized: now 1786149995386838 us; error 47 us; skew 500 ppm
Aug 08 00:46:35 dist-test-slave-kvfs krb5kdc[13416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:35.390079 13976 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:46:35.390661 13976 webserver.cc:460] Webserver started at http://127.20.105.67:40477/ using document root <none> and password file <none>
I20260808 00:46:35.390908 13976 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:35.390951 13976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:35.391059 13976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:35.392335 13976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "7c35bebdab3f4ec79141d9fc41885817"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "6c5f5ae03a6c6fe54a785af797e66e53"
server_key_iv: "c09eeb3009d28b6db533fb405edb1dbf"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.392714 13976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "7c35bebdab3f4ec79141d9fc41885817"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "6c5f5ae03a6c6fe54a785af797e66e53"
server_key_iv: "c09eeb3009d28b6db533fb405edb1dbf"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.394295 13976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:46:35.395236 14002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.395565 13976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:35.395682 13976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "7c35bebdab3f4ec79141d9fc41885817"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "6c5f5ae03a6c6fe54a785af797e66e53"
server_key_iv: "c09eeb3009d28b6db533fb405edb1dbf"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.395783 13976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:35.419332 13976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:35.422063 13976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:35.422358 13976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:35.422681 13976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:35.423085 13976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:35.423156 13976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.423209 13976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:35.423239 13976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.430537 13976 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:43403
I20260808 00:46:35.430584 14163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:43403 every 8 connection(s)
I20260808 00:46:35.431109 13976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:46:35 dist-test-slave-kvfs krb5kdc[13416](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149995, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:35.441397 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 13976
I20260808 00:46:35.441540 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:46:35.441805 20901 external_mini_cluster.cc:1428] Setting key 467570ca104645cf605270ddbdcc4479
I20260808 00:46:35.443166 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.432129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:33125 (local address 127.20.105.126:40081)
0808 00:46:35.432197 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.432203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.432468 (+ 265us) server_negotiation.cc:407] Connection header received
0808 00:46:35.433057 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.433059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.433072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.433132 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:35.434243 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.434828 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.435976 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.436312 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.436385 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.439412 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:35.439415 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:35.439415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:35.439423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:35.441110 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:35.441758 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:35.441760 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:35.441760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:35.441834 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:35.442221 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:35.442223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:35.442224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:35.442423 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:35.442512 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.442923 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.443057 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:46:35.443410 14167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.432115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:40081 (local address 127.20.105.67:33125)
0808 00:46:35.432301 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.432349 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:35.432948 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:35.433199 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:35.433204 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:35.433878 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:35.434133 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.434138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.434929 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.434930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.435860 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.435862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.436430 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.436431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.436577 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.437401 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:35.437413 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:35.439281 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:35.441207 (+ 1926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:35.441218 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:35.441221 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:35.441608 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:35.441935 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:35.441936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:35.441936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:35.442125 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:35.442519 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:35.442528 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:35.442750 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 00:46:35.442881 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:46:35.443099 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260808 00:46:35.443985 14164 heartbeater.cc:347] Connected to a master server at 127.20.105.126:40081
I20260808 00:46:35.444196 14164 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:35.444448 14164 heartbeater.cc:511] Master 127.20.105.126:40081 requested a full tablet report, sending...
I20260808 00:46:35.445065 13482 ts_manager.cc:194] Registered new tserver with Master: 7c35bebdab3f4ec79141d9fc41885817 (127.20.105.67:43403)
I20260808 00:46:35.445604 13482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:33125
I20260808 00:46:35.454211 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:46:35.464116 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.456170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45966 (local address 127.20.105.126:40081)
0808 00:46:35.456255 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.456257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.456370 (+ 113us) server_negotiation.cc:407] Connection header received
0808 00:46:35.456416 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.456417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.456430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.456495 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:35.457213 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.457774 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.458686 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.458906 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.458995 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.460711 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:35.460712 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:35.460712 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:35.460720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:35.462502 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:35.462979 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:35.462980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:35.462981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:35.463049 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:35.463339 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:35.463341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:35.463341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:35.463572 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:35.463686 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.463798 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.463915 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 00:46:35.466504 13482 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45966:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:46:35.468734 13482 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:46:35.476208 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.471813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:43921 (local address 127.0.0.1:45964)
0808 00:46:35.472063 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.472479 (+ 416us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:35.472600 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:35.472818 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:35.472822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:35.472824 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:35.472937 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.472940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.473993 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.473994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.475069 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.475072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.475786 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.475787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.475871 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.475886 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:46:35.475994 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 00:46:35.476098 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":95,"threads_started":1}
I20260808 00:46:35.476208 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.472004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45964 (local address 127.20.105.65:43921)
0808 00:46:35.472386 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.472392 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.472563 (+ 171us) server_negotiation.cc:407] Connection header received
0808 00:46:35.472697 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.472699 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.472720 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.472809 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:35.473033 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.473869 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.475174 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.475697 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.475787 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.475861 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.476044 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.476094 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":135,"threads_started":1}
I20260808 00:46:35.477172 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.472147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:36623 (local address 127.0.0.1:51570)
0808 00:46:35.472629 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.472639 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:35.472720 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:35.473014 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:35.473014 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:35.473017 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:35.473058 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.473059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.474506 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.474507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.475754 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.475757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.476905 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.476907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.476975 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.476980 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:35.477034 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:46:35.477083 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"mutex_wait_us":53,"thread_start_us":107,"threads_started":1}
I20260808 00:46:35.477291 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.472114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.20.105.67:43403)
0808 00:46:35.472760 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.472764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.472775 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:46:35.472818 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.472820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.472838 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.472918 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:35.473195 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.474373 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.475861 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.476789 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.476848 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.476896 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.477131 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.477177 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":109,"threads_started":1}
I20260808 00:46:35.477145 13670 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba54193426f4492bf3bb216d3562e12 (DEFAULT_TABLE table=test-table [id=08adb5ad64aa42d8acb322fa1ab85d33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:35.477510 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.472282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51570 (local address 127.20.105.66:36623)
0808 00:46:35.472481 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.472485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.472728 (+ 243us) server_negotiation.cc:407] Connection header received
0808 00:46:35.472793 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.472796 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.472816 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.472899 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:35.473149 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.474371 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.475863 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.476725 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.476802 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.476865 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.477365 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.477429 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":111,"threads_started":1}
I20260808 00:46:35.477650 13670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba54193426f4492bf3bb216d3562e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:35.477679 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.471863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:43403 (local address 127.0.0.1:34070)
0808 00:46:35.472338 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.472376 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:35.472453 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:35.473051 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:35.473052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:35.473056 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:35.473119 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.473120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.474506 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.474507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.475754 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.475757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.476938 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.476939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.476990 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.476993 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:46:35.477041 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:46:35.477083 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":73,"thread_start_us":138,"threads_started":1}
I20260808 00:46:35.478262 13862 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba54193426f4492bf3bb216d3562e12 (DEFAULT_TABLE table=test-table [id=08adb5ad64aa42d8acb322fa1ab85d33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:35.478463 14070 tablet_service.cc:1467] Processing CreateTablet for tablet 1ba54193426f4492bf3bb216d3562e12 (DEFAULT_TABLE table=test-table [id=08adb5ad64aa42d8acb322fa1ab85d33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:35.478806 13862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba54193426f4492bf3bb216d3562e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:35.478849 14070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ba54193426f4492bf3bb216d3562e12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:35.481904 14194 tablet_bootstrap.cc:492] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Bootstrap starting.
I20260808 00:46:35.482571 14197 tablet_bootstrap.cc:492] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Bootstrap starting.
I20260808 00:46:35.482832 14194 tablet_bootstrap.cc:654] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:35.483325 14197 tablet_bootstrap.cc:654] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:35.483443 14194 log.cc:824] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:35.483827 14196 tablet_bootstrap.cc:492] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277: Bootstrap starting.
I20260808 00:46:35.483978 14197 log.cc:824] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:35.484484 14194 tablet_bootstrap.cc:492] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: No bootstrap required, opened a new log
I20260808 00:46:35.484565 14194 ts_tablet_manager.cc:1397] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:46:35.484637 14196 tablet_bootstrap.cc:654] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:35.484830 14197 tablet_bootstrap.cc:492] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: No bootstrap required, opened a new log
I20260808 00:46:35.484920 14197 ts_tablet_manager.cc:1397] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:46:35.485250 14196 log.cc:824] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:35.486106 14196 tablet_bootstrap.cc:492] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277: No bootstrap required, opened a new log
I20260808 00:46:35.486184 14196 ts_tablet_manager.cc:1397] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:46:35.486940 14197 raft_consensus.cc:348] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.487141 14197 raft_consensus.cc:374] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:35.487017 14194 raft_consensus.cc:348] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.487190 14197 raft_consensus.cc:729] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c35bebdab3f4ec79141d9fc41885817, State: Initialized, Role: FOLLOWER
I20260808 00:46:35.487192 14194 raft_consensus.cc:374] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:35.487231 14194 raft_consensus.cc:729] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3077a24f8b1a4eecad667c4b74dc7ae8, State: Initialized, Role: FOLLOWER
I20260808 00:46:35.487345 14197 consensus_queue.cc:260] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.487375 14194 consensus_queue.cc:260] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.487702 14194 ts_tablet_manager.cc:1428] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:46:35.487705 14197 ts_tablet_manager.cc:1428] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:46:35.487743 13966 heartbeater.cc:503] Master 127.20.105.126:40081 was elected leader, sending a full tablet report...
I20260808 00:46:35.487862 14164 heartbeater.cc:503] Master 127.20.105.126:40081 was elected leader, sending a full tablet report...
I20260808 00:46:35.488570 14196 raft_consensus.cc:348] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.488770 14196 raft_consensus.cc:374] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:35.488819 14196 raft_consensus.cc:729] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f31e33869e394a75ad2b1798466aa277, State: Initialized, Role: FOLLOWER
I20260808 00:46:35.488956 14196 consensus_queue.cc:260] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.489276 14196 ts_tablet_manager.cc:1428] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:46:35.489316 13764 heartbeater.cc:503] Master 127.20.105.126:40081 was elected leader, sending a full tablet report...
W20260808 00:46:35.548197 13766 tablet.cc:2369] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:46:35.681851 14166 tablet.cc:2369] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:46:35.729260 13970 tablet.cc:2369] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:46:35.737596 14204 raft_consensus.cc:482] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:35.737789 14204 raft_consensus.cc:504] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.738315 14204 leader_election.cc:288] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f31e33869e394a75ad2b1798466aa277 (127.20.105.65:43921), 7c35bebdab3f4ec79141d9fc41885817 (127.20.105.67:43403)
I20260808 00:46:35.741816 13972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.738569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:43921 (local address 127.20.105.66:33927)
0808 00:46:35.738643 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.738659 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:35.738716 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:35.738975 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:35.738976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:35.738979 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:35.739089 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.739092 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.739876 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.739878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.740887 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.740890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.741516 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.741517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.741622 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.741627 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:35.741693 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:46:35.741735 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":47}
I20260808 00:46:35.741952 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.738697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:33927 (local address 127.20.105.65:43921)
0808 00:46:35.738772 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.738775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.738789 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:46:35.738837 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.738838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.738851 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.738918 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:35.739202 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.739778 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.741041 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.741419 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.741483 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.741493 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.741808 (+ 315us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.741869 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:46:35.741971 14207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.738590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:43403 (local address 127.20.105.66:51639)
0808 00:46:35.738768 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.738779 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:35.738856 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:35.739094 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:35.739094 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:35.739096 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:35.739210 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.739212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.739980 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.739981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.740878 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:35.740880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.741613 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.741614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:35.741798 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.741803 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:35.741862 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:46:35.741898 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":1,"thread_start_us":108,"threads_started":1}
I20260808 00:46:35.742048 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.738696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:51639 (local address 127.20.105.67:43403)
0808 00:46:35.738750 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.738752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.738920 (+ 168us) server_negotiation.cc:407] Connection header received
0808 00:46:35.738972 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.738974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.738989 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.739072 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:35.739304 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.739888 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.740989 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.741499 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.741582 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.741595 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.741896 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.741947 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:46:35.742370 13693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba54193426f4492bf3bb216d3562e12" candidate_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31e33869e394a75ad2b1798466aa277" is_pre_election: true
I20260808 00:46:35.742450 14099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba54193426f4492bf3bb216d3562e12" candidate_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c35bebdab3f4ec79141d9fc41885817" is_pre_election: true
I20260808 00:46:35.742630 13693 raft_consensus.cc:2454] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3077a24f8b1a4eecad667c4b74dc7ae8 in term 0.
I20260808 00:46:35.742643 14099 raft_consensus.cc:2454] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3077a24f8b1a4eecad667c4b74dc7ae8 in term 0.
I20260808 00:46:35.742956 13800 leader_election.cc:302] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3077a24f8b1a4eecad667c4b74dc7ae8, 7c35bebdab3f4ec79141d9fc41885817; no voters:
I20260808 00:46:35.743130 14204 raft_consensus.cc:2781] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:46:35.743216 14204 raft_consensus.cc:482] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:46:35.743253 14204 raft_consensus.cc:3037] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:35.744268 14204 raft_consensus.cc:504] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.744470 14204 leader_election.cc:288] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [CANDIDATE]: Term 1 election: Requested vote from peers f31e33869e394a75ad2b1798466aa277 (127.20.105.65:43921), 7c35bebdab3f4ec79141d9fc41885817 (127.20.105.67:43403)
I20260808 00:46:35.744681 14099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba54193426f4492bf3bb216d3562e12" candidate_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c35bebdab3f4ec79141d9fc41885817"
I20260808 00:46:35.744720 13693 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ba54193426f4492bf3bb216d3562e12" candidate_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f31e33869e394a75ad2b1798466aa277"
I20260808 00:46:35.744807 14099 raft_consensus.cc:3037] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:35.744807 13693 raft_consensus.cc:3037] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:35.745784 13693 raft_consensus.cc:2454] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3077a24f8b1a4eecad667c4b74dc7ae8 in term 1.
I20260808 00:46:35.745791 14099 raft_consensus.cc:2454] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3077a24f8b1a4eecad667c4b74dc7ae8 in term 1.
I20260808 00:46:35.745996 13800 leader_election.cc:302] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3077a24f8b1a4eecad667c4b74dc7ae8, 7c35bebdab3f4ec79141d9fc41885817; no voters:
I20260808 00:46:35.746165 14204 raft_consensus.cc:2781] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:35.746368 14204 raft_consensus.cc:686] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 1 LEADER]: Becoming Leader. State: Replica: 3077a24f8b1a4eecad667c4b74dc7ae8, State: Running, Role: LEADER
I20260808 00:46:35.746513 14204 consensus_queue.cc:237] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } }
I20260808 00:46:35.747939 13482 catalog_manager.cc:5495] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3077a24f8b1a4eecad667c4b74dc7ae8 (127.20.105.66). New cstate: current_term: 1 leader_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3077a24f8b1a4eecad667c4b74dc7ae8" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 } health_report { overall_health: UNKNOWN } } }
I20260808 00:46:35.766543 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.763604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51586 (local address 127.20.105.66:36623)
0808 00:46:35.763695 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:35.763698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:35.763707 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:46:35.763758 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:35.763759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:35.763784 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:35.763858 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:46:35.764118 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.764680 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.765544 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:35.765766 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:35.765836 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:35.766046 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:46:35.766197 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:46:35.766286 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:35.766322 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 00:46:35.766349 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:46:35.769142 14099 raft_consensus.cc:1261] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 1 FOLLOWER]: Refusing update from remote peer 3077a24f8b1a4eecad667c4b74dc7ae8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:35.769170 13693 raft_consensus.cc:1261] T 1ba54193426f4492bf3bb216d3562e12 P f31e33869e394a75ad2b1798466aa277 [term 1 FOLLOWER]: Refusing update from remote peer 3077a24f8b1a4eecad667c4b74dc7ae8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:35.769652 14208 consensus_queue.cc:1035] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f31e33869e394a75ad2b1798466aa277" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 43921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:35.769830 14204 consensus_queue.cc:1035] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c35bebdab3f4ec79141d9fc41885817" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:35.772975 14213 mvcc.cc:205] Tried to move back new op lower bound from 7316070382666506240 to 7316070382578675712. Current Snapshot: MvccSnapshot[applied={T|T < 7316070382666506240}]
I20260808 00:46:35.776331 13482 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:35.776481 13482 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45966:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:35.777434 13482 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52028e5ecb014ab9893e0545eefb162d: Sending DeleteTablet for 3 replicas of tablet 1ba54193426f4492bf3bb216d3562e12
I20260808 00:46:35.777855 13862 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba54193426f4492bf3bb216d3562e12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:35 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:51570
I20260808 00:46:35.777858 14070 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba54193426f4492bf3bb216d3562e12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:35 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:34070
I20260808 00:46:35.777856 13670 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ba54193426f4492bf3bb216d3562e12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:35 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:45964
I20260808 00:46:35.778215 14234 tablet_replica.cc:330] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: stopping tablet replica
I20260808 00:46:35.778357 14234 raft_consensus.cc:2229] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:35.778366 14235 tablet_replica.cc:330] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: stopping tablet replica
I20260808 00:46:35.778498 14234 raft_consensus.cc:2258] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:35.778625 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 13579
I20260808 00:46:35.778601 14235 raft_consensus.cc:2229] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:46:35.779805 14235 raft_consensus.cc:2258] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:35.783104 14235 ts_tablet_manager.cc:1905] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:35.792971 14235 ts_tablet_manager.cc:1918] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:35.793139 14235 log.cc:1196] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/1ba54193426f4492bf3bb216d3562e12
I20260808 00:46:35.793483 14235 ts_tablet_manager.cc:1939] T 1ba54193426f4492bf3bb216d3562e12 P 3077a24f8b1a4eecad667c4b74dc7ae8: Deleting consensus metadata
I20260808 00:46:35.794183 13458 catalog_manager.cc:4841] TS 3077a24f8b1a4eecad667c4b74dc7ae8 (127.20.105.66:36623): tablet 1ba54193426f4492bf3bb216d3562e12 (table test-table [id=08adb5ad64aa42d8acb322fa1ab85d33]) successfully deleted
W20260808 00:46:35.798700 13458 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:43921 (error 108)
I20260808 00:46:35.799060 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 13770
I20260808 00:46:35.799531 14187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:35.799364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:43921 (local address 127.0.0.1:45974)
0808 00:46:35.799426 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:35.799484 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:43921: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:46:35.799860 13458 catalog_manager.cc:4584] TS f31e33869e394a75ad2b1798466aa277 (127.20.105.65:43921): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ba54193426f4492bf3bb216d3562e12: Network error: Client connection negotiation failed: client connection to 127.20.105.65:43921: connect: Connection refused (error 111)
I20260808 00:46:35.799328 14234 ts_tablet_manager.cc:1905] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:35.805385 14234 ts_tablet_manager.cc:1918] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:35.805518 14234 log.cc:1196] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/1ba54193426f4492bf3bb216d3562e12
I20260808 00:46:35.805909 14234 ts_tablet_manager.cc:1939] T 1ba54193426f4492bf3bb216d3562e12 P 7c35bebdab3f4ec79141d9fc41885817: Deleting consensus metadata
I20260808 00:46:35.806695 13461 catalog_manager.cc:4841] TS 7c35bebdab3f4ec79141d9fc41885817 (127.20.105.67:43403): tablet 1ba54193426f4492bf3bb216d3562e12 (table test-table [id=08adb5ad64aa42d8acb322fa1ab85d33]) successfully deleted
I20260808 00:46:35.813809 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 13976
I20260808 00:46:35.828529 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 13434
2026-08-08T00:46:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2611 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T00:46:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:35Z Disabled control of system clock
I20260808 00:46:35.875723 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:42969
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:42969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:35.974107 14243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:35.974392 14243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:35.974416 14243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:35.975617 14243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:35.975659 14243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:35.975672 14243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:35.975685 14243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:35.975698 14243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:35.977228 14243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:42969
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:42969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:35.977520 14243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:35.977851 14243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:35.983723 14253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:35.983729 14256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:35.983783 14252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:35.984059 14243 server_base.cc:1034] running on GCE node
I20260808 00:46:35.984419 14243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:35.984803 14243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:35.986011 14243 hybrid_clock.cc:648] HybridClock initialized: now 1786149995985988 us; error 45 us; skew 500 ppm
I20260808 00:46:35.987630 14243 webserver.cc:460] Webserver started at http://127.20.105.126:44935/ using document root <none> and password file <none>
I20260808 00:46:35.987962 14243 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:35.988072 14243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:35.988250 14243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:35.989547 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "9c97079172d64b13b1f0abab5fa249a3"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "53d8b7551e09a0a80eae9e78cd68c394"
server_key_iv: "d0d79a20c3f648df7de204c5a3cdcf0e"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.989976 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "9c97079172d64b13b1f0abab5fa249a3"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "53d8b7551e09a0a80eae9e78cd68c394"
server_key_iv: "d0d79a20c3f648df7de204c5a3cdcf0e"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.991523 14243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:46:35.992462 14270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:35.992704 14243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:46:35.992833 14243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "9c97079172d64b13b1f0abab5fa249a3"
format_stamp: "Formatted at 2026-08-08 00:46:35 on dist-test-slave-kvfs"
server_key: "53d8b7551e09a0a80eae9e78cd68c394"
server_key_iv: "d0d79a20c3f648df7de204c5a3cdcf0e"
server_key_version: "encryptionkey@0"
I20260808 00:46:35.992920 14243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:36.014536 14243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:36.014997 14243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:36.015190 14243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:36.020650 14243 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:42969
I20260808 00:46:36.020730 14352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:42969 every 8 connection(s)
I20260808 00:46:36.021314 14243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:36.022130 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 14243
I20260808 00:46:36.022298 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:36.022562 20901 external_mini_cluster.cc:1428] Setting key 79f29d7f34238a822484b452e742e9be
I20260808 00:46:36.022796 14353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:36.027197 14353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3: Bootstrap starting.
I20260808 00:46:36.028151 14353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:36.028635 14353 log.cc:824] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:36.029527 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.023299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.20.105.126:42969)
0808 00:46:36.023534 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.023544 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.023575 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:46:36.024320 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.024331 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.024414 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.024559 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:36.024890 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.025884 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.027059 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.027644 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.027757 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.028697 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:36.028702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:36.028702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:36.028984 (+ 282us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:36.029026 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:36.029039 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:36.029119 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.029208 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.029307 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
I20260808 00:46:36.029603 14353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3: No bootstrap required, opened a new log
I20260808 00:46:36.031910 14353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c97079172d64b13b1f0abab5fa249a3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42969 } }
I20260808 00:46:36.032183 14353 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:36.032239 14353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c97079172d64b13b1f0abab5fa249a3, State: Initialized, Role: FOLLOWER
I20260808 00:46:36.032387 14353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c97079172d64b13b1f0abab5fa249a3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42969 } }
I20260808 00:46:36.032496 14353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:36.032543 14353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:36.032603 14353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:36.033512 14353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c97079172d64b13b1f0abab5fa249a3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42969 } }
I20260808 00:46:36.033664 14353 leader_election.cc:302] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c97079172d64b13b1f0abab5fa249a3; no voters:
I20260808 00:46:36.033896 14353 leader_election.cc:288] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:36.034147 14360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:36.034314 14353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:36.034385 14360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [term 1 LEADER]: Becoming Leader. State: Replica: 9c97079172d64b13b1f0abab5fa249a3, State: Running, Role: LEADER
I20260808 00:46:36.034495 14360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c97079172d64b13b1f0abab5fa249a3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42969 } }
I20260808 00:46:36.034929 14360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c97079172d64b13b1f0abab5fa249a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c97079172d64b13b1f0abab5fa249a3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42969 } } }
I20260808 00:46:36.035056 14360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:36.035213 14360 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c97079172d64b13b1f0abab5fa249a3. Latest consensus state: current_term: 1 leader_uuid: "9c97079172d64b13b1f0abab5fa249a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c97079172d64b13b1f0abab5fa249a3" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 42969 } } }
I20260808 00:46:36.035274 14360 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:36.035403 14369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:36.036059 14369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:36.038300 14369 catalog_manager.cc:1347] Generated new cluster ID: 2cbfc0fa39e443b59ce79cd871f5540e
I20260808 00:46:36.038347 14369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:36.048257 14369 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:36.048913 14369 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:36.060415 14369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3: Generated new TSK 0
I20260808 00:46:36.060734 14369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:36.066900 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:42969
--builtin_ntp_servers=127.20.105.84:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:36.166466 14388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:36.166764 14388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:36.166800 14388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:36.168080 14388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:36.168144 14388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:36.168186 14388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:46:36.169649 14388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:36.169974 14388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:36.170356 14388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:36.177767 14395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:36.177767 14396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:36.178110 14388 server_base.cc:1034] running on GCE node
W20260808 00:46:36.178242 14400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:36.178498 14388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:36.178900 14388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:36.180100 14388 hybrid_clock.cc:648] HybridClock initialized: now 1786149996180083 us; error 44 us; skew 500 ppm
I20260808 00:46:36.181936 14388 webserver.cc:460] Webserver started at http://127.20.105.65:34945/ using document root <none> and password file <none>
I20260808 00:46:36.182281 14388 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:36.182358 14388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:36.182526 14388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:36.184355 14388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "0f98620ade2d4643a36980a3eebcb416"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "5466d300811a743e5fb3b9b6bcbb1b2e"
server_key_iv: "182d575f7ccd53c4a88d7ee72d4546bf"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.184921 14388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "0f98620ade2d4643a36980a3eebcb416"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "5466d300811a743e5fb3b9b6bcbb1b2e"
server_key_iv: "182d575f7ccd53c4a88d7ee72d4546bf"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.186641 14388 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:46:36.187563 14409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.187785 14388 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:36.187947 14388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "0f98620ade2d4643a36980a3eebcb416"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "5466d300811a743e5fb3b9b6bcbb1b2e"
server_key_iv: "182d575f7ccd53c4a88d7ee72d4546bf"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.188079 14388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:36.201097 14388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:36.201602 14388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:36.201817 14388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:36.202172 14388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:36.202689 14388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:36.202775 14388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.202854 14388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:36.202893 14388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.212059 14388 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:41285
I20260808 00:46:36.212179 14574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:41285 every 8 connection(s)
I20260808 00:46:36.212735 14388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:46:36.215463 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 14388
I20260808 00:46:36.215639 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:46:36.215880 20901 external_mini_cluster.cc:1428] Setting key 7e4cf92aab305e147599939c96913104
I20260808 00:46:36.217423 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:42969
--builtin_ntp_servers=127.20.105.84:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 00:46:36.221546 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.213905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:37549 (local address 127.20.105.126:42969)
0808 00:46:36.214000 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.214002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.214405 (+ 403us) server_negotiation.cc:407] Connection header received
0808 00:46:36.215204 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.215207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.215226 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.215295 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:36.216685 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.217557 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.219356 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.219711 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.219820 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.221052 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:36.221054 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:36.221055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:36.221063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:36.221085 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:36.221090 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:36.221157 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.221345 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.221415 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:46:36.221791 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.213855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:42969 (local address 127.20.105.65:37549)
0808 00:46:36.214230 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.214294 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:36.214999 (+ 705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:36.215393 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:36.215399 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:36.215412 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:36.216545 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.216556 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.217700 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.217702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.219139 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.219144 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.219851 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.219853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.220106 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.220859 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:36.220859 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:36.220900 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:36.220935 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:36.221223 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:36.221231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:36.221235 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:36.221325 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 00:46:36.221453 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":139,"threads_started":1}
I20260808 00:46:36.222460 14576 heartbeater.cc:347] Connected to a master server at 127.20.105.126:42969
I20260808 00:46:36.222640 14576 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:36.222913 14576 heartbeater.cc:511] Master 127.20.105.126:42969 requested a full tablet report, sending...
I20260808 00:46:36.223865 14298 ts_manager.cc:194] Registered new tserver with Master: 0f98620ade2d4643a36980a3eebcb416 (127.20.105.65:41285)
I20260808 00:46:36.225051 14298 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.65:37549
W20260808 00:46:36.325958 14580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:36.326303 14580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:36.326334 14580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:36.328004 14580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:36.328127 14580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:36.328162 14580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:46:36.329947 14580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:36.330354 14580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:36.330840 14580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:36.338438 14592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:36.338438 14589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:36.338701 14590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:36.338898 14580 server_base.cc:1034] running on GCE node
I20260808 00:46:36.339232 14580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:36.339612 14580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:36.340794 14580 hybrid_clock.cc:648] HybridClock initialized: now 1786149996340767 us; error 48 us; skew 500 ppm
I20260808 00:46:36.342501 14580 webserver.cc:460] Webserver started at http://127.20.105.66:38349/ using document root <none> and password file <none>
I20260808 00:46:36.342880 14580 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:36.342980 14580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:36.343178 14580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:36.344712 14580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "b1ff792cdcc7f417bac29499ec109494"
server_key_iv: "90ed3ed2804cc7aa11e7cf6c130c9115"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.345199 14580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "b1ff792cdcc7f417bac29499ec109494"
server_key_iv: "90ed3ed2804cc7aa11e7cf6c130c9115"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.347115 14580 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:46:36.348346 14601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.348749 14580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:36.348919 14580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "b1ff792cdcc7f417bac29499ec109494"
server_key_iv: "90ed3ed2804cc7aa11e7cf6c130c9115"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.349025 14580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:36.359292 14580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:36.359776 14580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:36.359966 14580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:36.360351 14580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:36.360814 14580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:36.360872 14580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.360918 14580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:36.360950 14580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.368232 14580 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:39701
I20260808 00:46:36.368286 14776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:39701 every 8 connection(s)
I20260808 00:46:36.368722 14580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:46:36.375571 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.369690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:51587 (local address 127.20.105.126:42969)
0808 00:46:36.369842 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.369844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.370083 (+ 239us) server_negotiation.cc:407] Connection header received
0808 00:46:36.370732 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.370733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.370747 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.370831 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:36.371788 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.372611 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.373791 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.374011 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.374108 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.375192 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:36.375194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:36.375194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:36.375200 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:36.375218 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:36.375223 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:36.375280 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.375444 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.375477 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260808 00:46:36.375880 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.369648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:42969 (local address 127.20.105.66:51587)
0808 00:46:36.369882 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.369937 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:36.370604 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:36.370941 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:36.370946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:36.370957 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:36.371659 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.371665 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.372724 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.372725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.373697 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.373700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.374138 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.374139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.374299 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.375011 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:36.375011 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:36.375050 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:36.375083 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:36.375314 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:36.375321 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:36.375325 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:36.375406 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 00:46:36.375542 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":108,"threads_started":1}
I20260808 00:46:36.376600 14777 heartbeater.cc:347] Connected to a master server at 127.20.105.126:42969
I20260808 00:46:36.376748 14777 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:36.377023 14777 heartbeater.cc:511] Master 127.20.105.126:42969 requested a full tablet report, sending...
I20260808 00:46:36.377481 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 14580
I20260808 00:46:36.377563 14298 ts_manager.cc:194] Registered new tserver with Master: 0aa2efc43c5d4549805ba0f6e68dc9ba (127.20.105.66:39701)
I20260808 00:46:36.377616 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:46:36.378000 20901 external_mini_cluster.cc:1428] Setting key 9bd55306f6edde3d90e8beb3c63abebe
I20260808 00:46:36.378041 14298 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.66:51587
I20260808 00:46:36.379482 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:42969
--builtin_ntp_servers=127.20.105.84:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:36.483786 14781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:36.484158 14781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:36.484205 14781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:36.485483 14781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:36.485570 14781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:36.485620 14781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:46:36.487133 14781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:42969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:36.487476 14781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:36.487895 14781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:36.494586 14791 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:36.494578 14787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:36.494582 14788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:36.494877 14781 server_base.cc:1034] running on GCE node
I20260808 00:46:36.495177 14781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:36.495514 14781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:36.496706 14781 hybrid_clock.cc:648] HybridClock initialized: now 1786149996496694 us; error 43 us; skew 500 ppm
I20260808 00:46:36.498577 14781 webserver.cc:460] Webserver started at http://127.20.105.67:35953/ using document root <none> and password file <none>
I20260808 00:46:36.498828 14781 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:36.498874 14781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:36.498991 14781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:36.500353 14781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "ac299c555db0458fa667146d4a7158fc"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "3bf9f42a39266d4e3373d508e2631f4d"
server_key_iv: "717ede3dd2ef673e9562b440b9a2f0ea"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.500749 14781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "ac299c555db0458fa667146d4a7158fc"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "3bf9f42a39266d4e3373d508e2631f4d"
server_key_iv: "717ede3dd2ef673e9562b440b9a2f0ea"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.502323 14781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:46:36.503285 14799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.503634 14781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:36.503777 14781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "ac299c555db0458fa667146d4a7158fc"
format_stamp: "Formatted at 2026-08-08 00:46:36 on dist-test-slave-kvfs"
server_key: "3bf9f42a39266d4e3373d508e2631f4d"
server_key_iv: "717ede3dd2ef673e9562b440b9a2f0ea"
server_key_version: "encryptionkey@0"
I20260808 00:46:36.503870 14781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:36.515273 14781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:36.515718 14781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:36.515900 14781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:36.516273 14781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:36.516782 14781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:36.516844 14781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.516883 14781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:36.516911 14781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:36.524211 14781 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:40511
I20260808 00:46:36.524267 14954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:40511 every 8 connection(s)
I20260808 00:46:36.524708 14781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:46:36.528223 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 14781
I20260808 00:46:36.528357 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:46:36.528635 20901 external_mini_cluster.cc:1428] Setting key 11d3de00130c47641959ff22c8493567
I20260808 00:46:36.531808 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.525630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:55565 (local address 127.20.105.126:42969)
0808 00:46:36.525714 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.525716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.526046 (+ 330us) server_negotiation.cc:407] Connection header received
0808 00:46:36.526692 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.526694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.526708 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.526782 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:36.527664 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.528390 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.530076 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.530430 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.530504 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.531451 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:36.531453 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:36.531453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:36.531458 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:36.531476 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:36.531480 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:36.531535 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.531700 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.531730 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 00:46:36.532173 14959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.525604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:42969 (local address 127.20.105.67:55565)
0808 00:46:36.525850 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.525902 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:36.526537 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:36.526898 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:36.526903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:36.526915 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:36.527522 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.527528 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.528519 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.528521 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.529977 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.529981 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.530530 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.530531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.530667 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.531298 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:36.531299 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:36.531334 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:36.531364 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:36.531571 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:36.531578 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:36.531582 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:36.531670 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:46:36.531784 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":125,"threads_started":1}
I20260808 00:46:36.532938 14955 heartbeater.cc:347] Connected to a master server at 127.20.105.126:42969
I20260808 00:46:36.533116 14955 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:36.533424 14955 heartbeater.cc:511] Master 127.20.105.126:42969 requested a full tablet report, sending...
I20260808 00:46:36.533913 14298 ts_manager.cc:194] Registered new tserver with Master: ac299c555db0458fa667146d4a7158fc (127.20.105.67:40511)
I20260808 00:46:36.534391 14298 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.67:55565
I20260808 00:46:36.541612 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:46:36.547320 14357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.543442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59682 (local address 127.20.105.126:42969)
0808 00:46:36.543549 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.543552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.543667 (+ 115us) server_negotiation.cc:407] Connection header received
0808 00:46:36.543734 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.543735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.543748 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.543814 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:36.544121 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.544750 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.545820 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.546056 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.546127 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.547045 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:36.547047 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:36.547048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:36.547053 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:36.547071 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:36.547074 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:36.547129 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.547197 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.547227 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 00:46:36.549311 14298 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:46:36.551302 14298 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:46:36.558750 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.554609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50546 (local address 127.20.105.67:40511)
0808 00:46:36.555096 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.555101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.555112 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:46:36.555156 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.555158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.555174 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.555251 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:36.555480 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.556636 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.557719 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.558287 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.558386 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.558503 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.558595 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.558640 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":98,"threads_started":1}
I20260808 00:46:36.558736 14971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.554463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:40511 (local address 127.0.0.1:50546)
0808 00:46:36.554868 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.554881 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:36.554945 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:36.555330 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:36.555331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:36.555334 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:36.555386 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.555388 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.556725 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.556726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.557618 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.557621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.558393 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.558395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.558477 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.558494 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 00:46:36.558577 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:46:36.558640 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":53,"threads_started":1}
I20260808 00:46:36.558872 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.554199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39701 (local address 127.0.0.1:57640)
0808 00:46:36.554380 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.554438 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:36.554525 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:36.555069 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:36.555070 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:36.555073 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:36.555159 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.555161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.556284 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.556286 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.557639 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.557642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.558626 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.558627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.558694 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.558699 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:36.558732 (+ 33us) client_negotiation.cc:240] Negotiation successful
0808 00:46:36.558782 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":3,"thread_start_us":89,"threads_started":1}
I20260808 00:46:36.559098 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.554327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33134 (local address 127.20.105.65:41285)
0808 00:46:36.554687 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.554694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.554708 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:46:36.554784 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.554788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.554808 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.554924 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:36.555244 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.556496 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.557775 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.558514 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.558593 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.558690 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.558920 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.558988 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":122,"threads_started":1}
I20260808 00:46:36.559429 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.554593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57640 (local address 127.20.105.66:39701)
0808 00:46:36.554844 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.554850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.554863 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:46:36.554918 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.554920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.554939 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.555247 (+ 308us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:36.555325 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.556193 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.557751 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.558521 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.559070 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.559185 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.559253 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.559302 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":123,"threads_started":1}
I20260808 00:46:36.559330 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.554201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:41285 (local address 127.0.0.1:33134)
0808 00:46:36.554644 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.554655 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:36.554700 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:36.555044 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:36.555047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:36.555051 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:36.555147 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.555150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.556590 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.556591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.557682 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.557685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.558724 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.558725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.558777 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.558780 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:46:36.558825 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:46:36.558859 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":28,"thread_start_us":105,"threads_started":1}
I20260808 00:46:36.560204 14672 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecf5b63e1c94d12b2617510ace3b5a1 (DEFAULT_TABLE table=test-table [id=e53bbd5500d24dd88ec20e29bc32a50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:36.560205 14866 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecf5b63e1c94d12b2617510ace3b5a1 (DEFAULT_TABLE table=test-table [id=e53bbd5500d24dd88ec20e29bc32a50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:36.560676 14866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecf5b63e1c94d12b2617510ace3b5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:36.560676 14672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecf5b63e1c94d12b2617510ace3b5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:36.564673 14489 tablet_service.cc:1467] Processing CreateTablet for tablet 5ecf5b63e1c94d12b2617510ace3b5a1 (DEFAULT_TABLE table=test-table [id=e53bbd5500d24dd88ec20e29bc32a50d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:36.564844 14979 tablet_bootstrap.cc:492] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: Bootstrap starting.
I20260808 00:46:36.565192 14489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ecf5b63e1c94d12b2617510ace3b5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:36.565552 14978 tablet_bootstrap.cc:492] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Bootstrap starting.
I20260808 00:46:36.566494 14978 tablet_bootstrap.cc:654] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:36.566891 14978 log.cc:824] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:36.567646 14979 tablet_bootstrap.cc:654] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:36.567924 14978 tablet_bootstrap.cc:492] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: No bootstrap required, opened a new log
I20260808 00:46:36.568045 14978 ts_tablet_manager.cc:1397] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:46:36.568233 14979 log.cc:824] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:36.569268 14979 tablet_bootstrap.cc:492] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: No bootstrap required, opened a new log
I20260808 00:46:36.569356 14979 ts_tablet_manager.cc:1397] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 00:46:36.570648 14983 tablet_bootstrap.cc:492] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416: Bootstrap starting.
I20260808 00:46:36.571015 14978 raft_consensus.cc:348] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.571226 14978 raft_consensus.cc:374] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:36.571297 14978 raft_consensus.cc:729] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0aa2efc43c5d4549805ba0f6e68dc9ba, State: Initialized, Role: FOLLOWER
I20260808 00:46:36.571576 14983 tablet_bootstrap.cc:654] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:36.571534 14978 consensus_queue.cc:260] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.571902 14978 ts_tablet_manager.cc:1428] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 00:46:36.571979 14983 log.cc:824] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:36.572095 14777 heartbeater.cc:503] Master 127.20.105.126:42969 was elected leader, sending a full tablet report...
I20260808 00:46:36.572728 14979 raft_consensus.cc:348] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.572943 14979 raft_consensus.cc:374] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:36.572985 14979 raft_consensus.cc:729] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac299c555db0458fa667146d4a7158fc, State: Initialized, Role: FOLLOWER
I20260808 00:46:36.573107 14983 tablet_bootstrap.cc:492] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416: No bootstrap required, opened a new log
I20260808 00:46:36.573184 14983 ts_tablet_manager.cc:1397] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:46:36.573167 14979 consensus_queue.cc:260] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.573488 14979 ts_tablet_manager.cc:1428] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:46:36.573545 14955 heartbeater.cc:503] Master 127.20.105.126:42969 was elected leader, sending a full tablet report...
I20260808 00:46:36.575956 14983 raft_consensus.cc:348] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.576164 14983 raft_consensus.cc:374] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:36.576198 14983 raft_consensus.cc:729] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f98620ade2d4643a36980a3eebcb416, State: Initialized, Role: FOLLOWER
I20260808 00:46:36.576290 14983 consensus_queue.cc:260] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.576532 14983 ts_tablet_manager.cc:1428] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:46:36.576633 14576 heartbeater.cc:503] Master 127.20.105.126:42969 was elected leader, sending a full tablet report...
W20260808 00:46:36.619451 14779 tablet.cc:2369] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:46:36.670399 14984 raft_consensus.cc:482] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:36.670639 14984 raft_consensus.cc:504] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.671170 14984 leader_election.cc:288] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f98620ade2d4643a36980a3eebcb416 (127.20.105.65:41285), ac299c555db0458fa667146d4a7158fc (127.20.105.67:40511)
I20260808 00:46:36.674621 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.671565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:48859 (local address 127.20.105.65:41285)
0808 00:46:36.671624 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.671627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.671638 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:46:36.671679 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.671680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.671695 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.671762 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:36.672006 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.672644 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.673761 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.674192 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.674289 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.674299 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.674479 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.674532 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:46:36.674624 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.671789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:55643 (local address 127.20.105.67:40511)
0808 00:46:36.671860 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.671863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.671900 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:46:36.672050 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.672051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.672060 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.672129 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:36.672318 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.672828 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.673801 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.674205 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.674289 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.674299 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.674486 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.674537 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:46:36.674630 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.671444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:41285 (local address 127.20.105.66:48859)
0808 00:46:36.671523 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.671538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:36.671601 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:36.671773 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:36.671775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:36.671777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:36.671893 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.671896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.672745 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.672747 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.673652 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.673654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.674289 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.674290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.674383 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.674387 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:36.674458 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:46:36.674515 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":10}
I20260808 00:46:36.674645 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.671611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:40511 (local address 127.20.105.66:55643)
0808 00:46:36.671838 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.671847 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:36.671908 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:36.672134 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:36.672135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:36.672137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:36.672229 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.672233 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.672924 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.672925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.673689 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:36.673691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.674303 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.674304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:36.674399 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.674402 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:46:36.674466 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 00:46:36.674516 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":3,"thread_start_us":151,"threads_started":1}
I20260808 00:46:36.675076 14892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecf5b63e1c94d12b2617510ace3b5a1" candidate_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac299c555db0458fa667146d4a7158fc" is_pre_election: true
I20260808 00:46:36.675082 14516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecf5b63e1c94d12b2617510ace3b5a1" candidate_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f98620ade2d4643a36980a3eebcb416" is_pre_election: true
I20260808 00:46:36.675276 14892 raft_consensus.cc:2454] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aa2efc43c5d4549805ba0f6e68dc9ba in term 0.
I20260808 00:46:36.675277 14516 raft_consensus.cc:2454] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0aa2efc43c5d4549805ba0f6e68dc9ba in term 0.
I20260808 00:46:36.675547 14602 leader_election.cc:302] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aa2efc43c5d4549805ba0f6e68dc9ba, ac299c555db0458fa667146d4a7158fc; no voters:
I20260808 00:46:36.675724 14984 raft_consensus.cc:2781] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:46:36.675805 14984 raft_consensus.cc:482] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:46:36.675827 14984 raft_consensus.cc:3037] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:36.676806 14984 raft_consensus.cc:504] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.676976 14984 leader_election.cc:288] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [CANDIDATE]: Term 1 election: Requested vote from peers 0f98620ade2d4643a36980a3eebcb416 (127.20.105.65:41285), ac299c555db0458fa667146d4a7158fc (127.20.105.67:40511)
I20260808 00:46:36.677184 14892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecf5b63e1c94d12b2617510ace3b5a1" candidate_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac299c555db0458fa667146d4a7158fc"
I20260808 00:46:36.677217 14516 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ecf5b63e1c94d12b2617510ace3b5a1" candidate_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f98620ade2d4643a36980a3eebcb416"
I20260808 00:46:36.677286 14892 raft_consensus.cc:3037] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:36.677292 14516 raft_consensus.cc:3037] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:36.678184 14892 raft_consensus.cc:2454] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aa2efc43c5d4549805ba0f6e68dc9ba in term 1.
I20260808 00:46:36.678243 14516 raft_consensus.cc:2454] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0aa2efc43c5d4549805ba0f6e68dc9ba in term 1.
I20260808 00:46:36.678390 14602 leader_election.cc:302] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0aa2efc43c5d4549805ba0f6e68dc9ba, ac299c555db0458fa667146d4a7158fc; no voters:
I20260808 00:46:36.678508 14984 raft_consensus.cc:2781] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:36.678694 14984 raft_consensus.cc:686] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 1 LEADER]: Becoming Leader. State: Replica: 0aa2efc43c5d4549805ba0f6e68dc9ba, State: Running, Role: LEADER
I20260808 00:46:36.678817 14984 consensus_queue.cc:237] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } }
I20260808 00:46:36.679981 14297 catalog_manager.cc:5495] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 0aa2efc43c5d4549805ba0f6e68dc9ba (127.20.105.66). New cstate: current_term: 1 leader_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0aa2efc43c5d4549805ba0f6e68dc9ba" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 } health_report { overall_health: UNKNOWN } } }
I20260808 00:46:36.706897 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.703980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.20.105.66:39701)
0808 00:46:36.704068 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:36.704071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:36.704084 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:46:36.704147 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:36.704148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:36.704175 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:36.704252 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:46:36.704467 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.705054 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.705865 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:36.706075 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:36.706145 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:36.706369 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:46:36.706537 (+ 168us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:46:36.706606 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:36.706694 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:46:36.706719 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 00:46:36.709827 14516 raft_consensus.cc:1261] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416 [term 1 FOLLOWER]: Refusing update from remote peer 0aa2efc43c5d4549805ba0f6e68dc9ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:36.709827 14892 raft_consensus.cc:1261] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 1 FOLLOWER]: Refusing update from remote peer 0aa2efc43c5d4549805ba0f6e68dc9ba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:36.710318 14984 consensus_queue.cc:1035] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac299c555db0458fa667146d4a7158fc" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 40511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:36.710520 14989 consensus_queue.cc:1035] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f98620ade2d4643a36980a3eebcb416" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 41285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:46:36.713798 14577 tablet.cc:2369] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0f98620ade2d4643a36980a3eebcb416: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:46:36.717495 14298 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:36.717635 14298 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:36.718649 14298 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c97079172d64b13b1f0abab5fa249a3: Sending DeleteTablet for 3 replicas of tablet 5ecf5b63e1c94d12b2617510ace3b5a1
I20260808 00:46:36.719095 14866 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecf5b63e1c94d12b2617510ace3b5a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:36 UTC) from {username='slave'} at 127.0.0.1:50546
I20260808 00:46:36.719115 14672 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecf5b63e1c94d12b2617510ace3b5a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:36 UTC) from {username='slave'} at 127.0.0.1:57640
I20260808 00:46:36.719144 14489 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ecf5b63e1c94d12b2617510ace3b5a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:36 UTC) from {username='slave'} at 127.0.0.1:33134
I20260808 00:46:36.719408 15008 tablet_replica.cc:330] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: stopping tablet replica
I20260808 00:46:36.719548 15008 raft_consensus.cc:2229] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:36.719746 15008 raft_consensus.cc:2258] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:36.719851 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 14388
I20260808 00:46:36.719980 15006 tablet_replica.cc:330] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: stopping tablet replica
I20260808 00:46:36.748556 15006 raft_consensus.cc:2229] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:46:36.748795 15006 raft_consensus.cc:2258] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:36.750072 15008 ts_tablet_manager.cc:1905] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:36.751359 15006 ts_tablet_manager.cc:1905] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:36.752357 15008 ts_tablet_manager.cc:1918] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:36.752449 15008 log.cc:1196] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/5ecf5b63e1c94d12b2617510ace3b5a1
I20260808 00:46:36.752768 15008 ts_tablet_manager.cc:1939] T 5ecf5b63e1c94d12b2617510ace3b5a1 P ac299c555db0458fa667146d4a7158fc: Deleting consensus metadata
I20260808 00:46:36.753463 15006 ts_tablet_manager.cc:1918] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:36.753433 14271 catalog_manager.cc:4841] TS ac299c555db0458fa667146d4a7158fc (127.20.105.67:40511): tablet 5ecf5b63e1c94d12b2617510ace3b5a1 (table test-table [id=e53bbd5500d24dd88ec20e29bc32a50d]) successfully deleted
I20260808 00:46:36.753612 15006 log.cc:1196] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/5ecf5b63e1c94d12b2617510ace3b5a1
I20260808 00:46:36.753959 15006 ts_tablet_manager.cc:1939] T 5ecf5b63e1c94d12b2617510ace3b5a1 P 0aa2efc43c5d4549805ba0f6e68dc9ba: Deleting consensus metadata
I20260808 00:46:36.754521 14273 catalog_manager.cc:4841] TS 0aa2efc43c5d4549805ba0f6e68dc9ba (127.20.105.66:39701): tablet 5ecf5b63e1c94d12b2617510ace3b5a1 (table test-table [id=e53bbd5500d24dd88ec20e29bc32a50d]) successfully deleted
W20260808 00:46:36.760421 14271 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:41285 (error 108)
I20260808 00:46:36.761143 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:36.760966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:41285 (local address 127.0.0.1:33136)
0808 00:46:36.761038 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:36.761096 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:41285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 00:46:36.761253 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 14580
W20260808 00:46:36.767542 14271 catalog_manager.cc:4584] TS 0f98620ade2d4643a36980a3eebcb416 (127.20.105.65:41285): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ecf5b63e1c94d12b2617510ace3b5a1: Network error: Client connection negotiation failed: client connection to 127.20.105.65:41285: connect: Connection refused (error 111)
I20260808 00:46:36.774032 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 14781
I20260808 00:46:36.791214 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 14243
2026-08-08T00:46:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (958 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:36 dist-test-slave-kvfs krb5kdc[15019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:36 dist-test-slave-kvfs krb5kdc[15019](info): set up 2 sockets
Aug 08 00:46:36 dist-test-slave-kvfs krb5kdc[15019](info): commencing operation
krb5kdc: starting...
W20260808 00:46:38.101038 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:38 dist-test-slave-kvfs krb5kdc[15019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:46:38.264328 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33897
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:33897
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:38.369130 15039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:38.369459 15039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:38.369496 15039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:38.370792 15039 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:38.370879 15039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:38.370901 15039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:38.370923 15039 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:38.370952 15039 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:38.372761 15039 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:33897
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:33897
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:38.373126 15039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:38.373421 15039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:38.379593 15050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:38.379606 15053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:38.379662 15051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:38.379886 15039 server_base.cc:1034] running on GCE node
I20260808 00:46:38.380334 15039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:38.380695 15039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:38.381907 15039 hybrid_clock.cc:648] HybridClock initialized: now 1786149998381866 us; error 57 us; skew 500 ppm
Aug 08 00:46:38 dist-test-slave-kvfs krb5kdc[15019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:38.385512 15039 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:46:38.386174 15039 webserver.cc:460] Webserver started at http://127.20.105.126:44389/ using document root <none> and password file <none>
I20260808 00:46:38.386473 15039 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:38.386525 15039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:38.386713 15039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:38.388191 15039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "d451021e2016454f9fd5334daf27fd3a"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "c7b6cf90e605de7380843fb4e54d89bd"
server_key_iv: "35c3029ebbf60125a84f6d4dbb387181"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.388653 15039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "d451021e2016454f9fd5334daf27fd3a"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "c7b6cf90e605de7380843fb4e54d89bd"
server_key_iv: "35c3029ebbf60125a84f6d4dbb387181"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.390376 15039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:46:38.391325 15067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:38.391625 15039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:38.391767 15039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "d451021e2016454f9fd5334daf27fd3a"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "c7b6cf90e605de7380843fb4e54d89bd"
server_key_iv: "35c3029ebbf60125a84f6d4dbb387181"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.391858 15039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:38.399897 15039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:38.402588 15039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:38.402863 15039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:38.408730 15039 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:33897
I20260808 00:46:38.408766 15156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:33897 every 8 connection(s)
I20260808 00:46:38.409272 15039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:38.410482 15158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:38.413630 15158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a: Bootstrap starting.
I20260808 00:46:38.413693 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 15039
I20260808 00:46:38.413877 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:38.414134 20901 external_mini_cluster.cc:1428] Setting key ed9ce5bacc2ff459aaae159ecf67a397
I20260808 00:46:38.414659 15158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:38.415385 15158 log.cc:824] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:38.416481 15158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a: No bootstrap required, opened a new log
I20260808 00:46:38.419484 15158 raft_consensus.cc:348] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d451021e2016454f9fd5334daf27fd3a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33897 } }
I20260808 00:46:38.419719 15158 raft_consensus.cc:374] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:38.419770 15158 raft_consensus.cc:729] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d451021e2016454f9fd5334daf27fd3a, State: Initialized, Role: FOLLOWER
I20260808 00:46:38.419976 15158 consensus_queue.cc:260] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d451021e2016454f9fd5334daf27fd3a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33897 } }
I20260808 00:46:38.420106 15158 raft_consensus.cc:388] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:38.420151 15158 raft_consensus.cc:482] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:38.420207 15158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:38.421147 15158 raft_consensus.cc:504] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d451021e2016454f9fd5334daf27fd3a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33897 } }
I20260808 00:46:38.421306 15158 leader_election.cc:302] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d451021e2016454f9fd5334daf27fd3a; no voters:
I20260808 00:46:38.421553 15158 leader_election.cc:288] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:38.421579 15163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:38.421718 15163 raft_consensus.cc:686] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [term 1 LEADER]: Becoming Leader. State: Replica: d451021e2016454f9fd5334daf27fd3a, State: Running, Role: LEADER
I20260808 00:46:38.421937 15158 sys_catalog.cc:564] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:38.421887 15163 consensus_queue.cc:237] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d451021e2016454f9fd5334daf27fd3a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33897 } }
I20260808 00:46:38.422528 15168 sys_catalog.cc:455] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d451021e2016454f9fd5334daf27fd3a. Latest consensus state: current_term: 1 leader_uuid: "d451021e2016454f9fd5334daf27fd3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d451021e2016454f9fd5334daf27fd3a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33897 } } }
I20260808 00:46:38.422639 15168 sys_catalog.cc:458] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:38.422932 15164 sys_catalog.cc:455] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d451021e2016454f9fd5334daf27fd3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d451021e2016454f9fd5334daf27fd3a" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 33897 } } }
Aug 08 00:46:38 dist-test-slave-kvfs krb5kdc[15019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:38.423514 15164 sys_catalog.cc:458] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a [sys.catalog]: This master's current role is: LEADER
W20260808 00:46:38.424561 15188 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:46:38.424630 15188 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:46:38.424760 15176 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:38.424957 15176 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:38.428006 15176 catalog_manager.cc:1347] Generated new cluster ID: 25aa8d821c9648039a30a686e31939e3
I20260808 00:46:38.428161 15176 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:38.429502 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:38.414892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60044 (local address 127.20.105.126:33897)
0808 00:46:38.415192 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:38.415201 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:38.415236 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:46:38.416079 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:38.416091 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:38.416156 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:38.416305 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:38.417327 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.418567 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.419801 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.420495 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.420594 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:38.424341 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:38.424347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:38.424347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:38.424361 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:38.427218 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:38.427810 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:38.427814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:38.427815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:38.427957 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:38.428324 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:38.428326 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:38.428327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:38.428714 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:38.428925 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:38.429053 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 00:46:38.429231 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":130,"threads_started":1}
I20260808 00:46:38.439103 15176 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:38.439888 15176 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:38.446210 15176 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a: Generated new TSK 0
I20260808 00:46:38.446497 15176 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:46:38.491182 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33897
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:38.592187 15201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:38.592463 15201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:38.592492 15201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:38.593818 15201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:38.593873 15201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:38.593902 15201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:46:38.595636 15201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:38.595955 15201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:38.596273 15201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:38.602439 15206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:38.602391 15207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:38.602409 15210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:38.602977 15201 server_base.cc:1034] running on GCE node
I20260808 00:46:38.603274 15201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:38.603524 15201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:38.604701 15201 hybrid_clock.cc:648] HybridClock initialized: now 1786149998604683 us; error 39 us; skew 500 ppm
Aug 08 00:46:38 dist-test-slave-kvfs krb5kdc[15019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:38.608134 15201 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:46:38.608716 15201 webserver.cc:460] Webserver started at http://127.20.105.65:46099/ using document root <none> and password file <none>
I20260808 00:46:38.609053 15201 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:38.609112 15201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:38.609232 15201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:38.610594 15201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "f17336e931b34947b9d02f34da87e3ab"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "2e2e297cf5b5ee71903eba8a37304182"
server_key_iv: "4e3ee56c0bed8ce53faf045fb71cbd38"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.610986 15201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "f17336e931b34947b9d02f34da87e3ab"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "2e2e297cf5b5ee71903eba8a37304182"
server_key_iv: "4e3ee56c0bed8ce53faf045fb71cbd38"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.612706 15201 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 00:46:38.613653 15218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:38.613966 15201 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:38.614102 15201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "f17336e931b34947b9d02f34da87e3ab"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "2e2e297cf5b5ee71903eba8a37304182"
server_key_iv: "4e3ee56c0bed8ce53faf045fb71cbd38"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.614199 15201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:38.626165 15201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:38.628731 15201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:38.628927 15201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:38.629273 15201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:38.629669 15201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:38.629727 15201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:38.629776 15201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:38.629804 15201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:38.637339 15201 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:40183
I20260808 00:46:38.637387 15383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:40183 every 8 connection(s)
I20260808 00:46:38.637842 15201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:46:38.639555 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 15201
I20260808 00:46:38.639729 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:46:38.640009 20901 external_mini_cluster.cc:1428] Setting key 04040356df9fc45bba1490a01d1a6ba8
Aug 08 00:46:38 dist-test-slave-kvfs krb5kdc[15019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:38.652087 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:38.638881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:36591 (local address 127.20.105.126:33897)
0808 00:46:38.639019 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:38.639022 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:38.639171 (+ 149us) server_negotiation.cc:407] Connection header received
0808 00:46:38.640287 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:38.640288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:38.640310 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:38.640394 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:38.642118 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.642712 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.644489 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.644886 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.644987 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:38.648374 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:38.648376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:38.648377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:38.648388 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:38.650095 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:38.650744 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:38.650745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:38.650746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:38.650815 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:38.651183 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:38.651183 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:38.651184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:38.651329 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:38.651389 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:38.651820 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 00:46:38.651930 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 00:46:38.652397 15392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:38.638778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33897 (local address 127.20.105.65:36591)
0808 00:46:38.639011 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:38.639063 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:38.640007 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:38.640543 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:38.640554 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:38.641592 (+ 1038us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:38.641966 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:38.641977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.642835 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.642836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:38.644364 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:38.644368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.645041 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.645042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:38.645191 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:38.645958 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:38.645981 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:38.648214 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:38.650202 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:38.650215 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:38.650218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:38.650642 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:38.650886 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:38.650887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:38.650888 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:38.651089 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:38.651401 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:38.651410 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:38.651639 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 00:46:38.651774 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 00:46:38.652067 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
I20260808 00:46:38.653199 15389 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33897
I20260808 00:46:38.653332 15389 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:38.653584 15389 heartbeater.cc:511] Master 127.20.105.126:33897 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
I20260808 00:46:38.654302 15093 ts_manager.cc:194] Registered new tserver with Master: f17336e931b34947b9d02f34da87e3ab (127.20.105.65:40183)
I20260808 00:46:38.655189 15093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:36591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:46:38.675817 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33897
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:38.776254 15398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:38.776540 15398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:38.776592 15398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:38.777936 15398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:38.778021 15398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:38.778072 15398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:46:38.779814 15398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:38.780189 15398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:38.780498 15398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:38.787761 15408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:38.787743 15411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:38.787743 15409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:38.787786 15398 server_base.cc:1034] running on GCE node
I20260808 00:46:38.788300 15398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:38.788628 15398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:38.789812 15398 hybrid_clock.cc:648] HybridClock initialized: now 1786149998789791 us; error 40 us; skew 500 ppm
Aug 08 00:46:38 dist-test-slave-kvfs krb5kdc[15019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:38.792954 15398 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:46:38.793551 15398 webserver.cc:460] Webserver started at http://127.20.105.66:41347/ using document root <none> and password file <none>
I20260808 00:46:38.793855 15398 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:38.793927 15398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:38.794096 15398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:38.795384 15398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "871b349e80054938a91df0c05f86cb05"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "944ead8bc661592bdcfd2b87148a7fc3"
server_key_iv: "a034c36b44295b169c44f3a531957a1b"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.795810 15398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "871b349e80054938a91df0c05f86cb05"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "944ead8bc661592bdcfd2b87148a7fc3"
server_key_iv: "a034c36b44295b169c44f3a531957a1b"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.797506 15398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:38.798544 15420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:38.798801 15398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:38.798925 15398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "871b349e80054938a91df0c05f86cb05"
format_stamp: "Formatted at 2026-08-08 00:46:38 on dist-test-slave-kvfs"
server_key: "944ead8bc661592bdcfd2b87148a7fc3"
server_key_iv: "a034c36b44295b169c44f3a531957a1b"
server_key_version: "encryptionkey@0"
I20260808 00:46:38.799028 15398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:38.828382 15398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:38.831437 15398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:38.831692 15398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:38.832103 15398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:38.832579 15398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:38.832643 15398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:38.832696 15398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:38.832742 15398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:38.840924 15398 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:39965
I20260808 00:46:38.840975 15595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:39965 every 8 connection(s)
I20260808 00:46:38.841463 15398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:46:38.844851 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 15398
I20260808 00:46:38.845145 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:46:38.845402 20901 external_mini_cluster.cc:1428] Setting key be6487a1ec4b7301f6d701ad3ea055e9
Aug 08 00:46:38 dist-test-slave-kvfs krb5kdc[15019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149998, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:38.855253 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:38.842510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:51767 (local address 127.20.105.126:33897)
0808 00:46:38.842589 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:38.842592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:38.842801 (+ 209us) server_negotiation.cc:407] Connection header received
0808 00:46:38.843443 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:38.843445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:38.843460 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:38.843548 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:38.844992 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.845739 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.847294 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.847672 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.847770 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:38.850779 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:38.850782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:38.850782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:38.850796 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:38.853174 (+ 2378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:38.854008 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:38.854010 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:38.854011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:38.854090 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:38.854391 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:38.854392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:38.854393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:38.854553 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:38.854634 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:38.855006 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 00:46:38.855126 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:46:38.855522 15598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:38.842409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33897 (local address 127.20.105.66:51767)
0808 00:46:38.842622 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:38.842679 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:38.843318 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:38.843661 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:38.843665 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:38.844483 (+ 818us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:38.844762 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:38.844768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.845977 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.845979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:38.847175 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:38.847179 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:38.847819 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:38.847821 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:38.847995 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:38.848802 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:38.848814 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:38.850625 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:38.853451 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:38.853467 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:38.853471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:38.853880 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:38.854166 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:38.854166 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:38.854167 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:38.854315 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:38.854632 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:38.854641 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:38.854847 (+ 206us) client_negotiation.cc:769] Sending connection context
0808 00:46:38.854971 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 00:46:38.855195 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":109,"threads_started":1}
I20260808 00:46:38.856346 15596 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33897
I20260808 00:46:38.856523 15596 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:38.856801 15596 heartbeater.cc:511] Master 127.20.105.126:33897 requested a full tablet report, sending...
I20260808 00:46:38.857546 15093 ts_manager.cc:194] Registered new tserver with Master: 871b349e80054938a91df0c05f86cb05 (127.20.105.66:39965)
I20260808 00:46:38.858004 15093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:51767
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:46:38.887501 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33897
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:38.993610 15602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:38.993944 15602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:38.993994 15602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:38.995497 15602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:38.995579 15602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:38.995633 15602 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:46:38.997596 15602 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:33897
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:38.997882 15602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:38.998162 15602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:39.004603 15624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.004603 15612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.004603 15614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:39.005055 15602 server_base.cc:1034] running on GCE node
I20260808 00:46:39.005360 15602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:39.005707 15602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:39.006915 15602 hybrid_clock.cc:648] HybridClock initialized: now 1786149999006899 us; error 36 us; skew 500 ppm
Aug 08 00:46:39 dist-test-slave-kvfs krb5kdc[15019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149999, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:39.010490 15602 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:46:39.011137 15602 webserver.cc:460] Webserver started at http://127.20.105.67:43763/ using document root <none> and password file <none>
I20260808 00:46:39.011451 15602 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:39.011528 15602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:39.011714 15602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:39.013109 15602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "44639a1d428648ce92a70da9c3cfed92"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "b91e7960d0a8fbd3abad71c22e3b7369"
server_key_iv: "df071ab9087b38bbc728b02fca929aab"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.013546 15602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "44639a1d428648ce92a70da9c3cfed92"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "b91e7960d0a8fbd3abad71c22e3b7369"
server_key_iv: "df071ab9087b38bbc728b02fca929aab"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.015195 15602 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:46:39.016155 15632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.016427 15602 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:46:39.016562 15602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "44639a1d428648ce92a70da9c3cfed92"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "b91e7960d0a8fbd3abad71c22e3b7369"
server_key_iv: "df071ab9087b38bbc728b02fca929aab"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.016659 15602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:39.043593 15602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:39.046605 15602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:39.046838 15602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:39.047422 15602 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:39.047991 15602 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:39.048087 15602 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.048141 15602 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:39.048166 15602 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.056942 15602 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:45841
I20260808 00:46:39.056995 15787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:45841 every 8 connection(s)
I20260808 00:46:39.057451 15602 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:46:39 dist-test-slave-kvfs krb5kdc[15019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149999, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:39.066855 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 15602
I20260808 00:46:39.067030 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:46:39.067305 20901 external_mini_cluster.cc:1428] Setting key 9334534afa82d1f981875be804115943
I20260808 00:46:39.069599 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.058544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:60167 (local address 127.20.105.126:33897)
0808 00:46:39.058632 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.058634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.058798 (+ 164us) server_negotiation.cc:407] Connection header received
0808 00:46:39.059452 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.059454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.059470 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.059552 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:39.060861 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.061469 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.062779 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.062989 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.063052 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.065982 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:39.065984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:39.065985 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:39.065993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:39.067876 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:39.068411 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:39.068414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:39.068414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:39.068486 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:39.068789 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:39.068790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:39.068791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:39.068938 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:39.069018 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.069362 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.069483 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:46:39.069849 15792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.058431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:33897 (local address 127.20.105.67:60167)
0808 00:46:39.058636 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.058688 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.059317 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.059642 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.059646 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.060459 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:39.060731 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.060737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.061591 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.061592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.062682 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.062685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.063085 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.063086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.063241 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.063958 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:39.063969 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:39.065875 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:39.067969 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:39.067980 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:39.067983 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:39.068310 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:39.068571 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:39.068572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:39.068572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:39.068710 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:39.069018 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:39.069027 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:39.069208 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.069325 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.069530 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":120,"threads_started":1}
I20260808 00:46:39.070425 15788 heartbeater.cc:347] Connected to a master server at 127.20.105.126:33897
I20260808 00:46:39.070559 15788 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:39.070811 15788 heartbeater.cc:511] Master 127.20.105.126:33897 requested a full tablet report, sending...
I20260808 00:46:39.071267 15093 ts_manager.cc:194] Registered new tserver with Master: 44639a1d428648ce92a70da9c3cfed92 (127.20.105.67:45841)
I20260808 00:46:39.071684 15093 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:60167
I20260808 00:46:39.080096 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:46:39.089943 15161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.082041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60060 (local address 127.20.105.126:33897)
0808 00:46:39.082163 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.082167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.082277 (+ 110us) server_negotiation.cc:407] Connection header received
0808 00:46:39.082382 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.082383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.082394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.082459 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:39.083169 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.083731 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.084635 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.084837 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.084929 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.086637 (+ 1708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:39.086639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:39.086640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:39.086650 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:39.088339 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:39.088809 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:39.088811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:39.088811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:39.088878 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:39.089249 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:39.089250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:39.089251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:39.089440 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:39.089589 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.089739 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.089813 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 00:46:39.092500 15093 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:46:39.094861 15093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:46:39.103025 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.098350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34440 (local address 127.20.105.66:39965)
0808 00:46:39.098639 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.098645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.098662 (+ 17us) server_negotiation.cc:407] Connection header received
0808 00:46:39.098729 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.098731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.098752 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.098863 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.099219 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.100344 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.101863 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.102519 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.102634 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.102762 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.102859 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.102909 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":124,"threads_started":1}
I20260808 00:46:39.103020 15809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.098183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39965 (local address 127.0.0.1:34440)
0808 00:46:39.098435 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.098488 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.098581 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.099021 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.099025 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.099029 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.099137 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.099141 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.100457 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.100458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.101764 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.101768 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.102636 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.102638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.102731 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.102749 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.102840 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.102907 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260808 00:46:39.103237 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.098499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37720 (local address 127.20.105.67:45841)
0808 00:46:39.098754 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.098760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.098972 (+ 212us) server_negotiation.cc:407] Connection header received
0808 00:46:39.099053 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.099056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.099075 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.099155 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.099284 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.100503 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.101933 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.102704 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.102776 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.102830 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.103095 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.103141 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":110,"threads_started":1}
I20260808 00:46:39.103394 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.098192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:45841 (local address 127.0.0.1:37720)
0808 00:46:39.098841 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.098851 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.098908 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.099165 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.099166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.099169 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.099225 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.099227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.100642 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.100643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.101832 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.101835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.102867 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.102869 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.102936 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.102941 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.102995 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.103044 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":14,"thread_start_us":114,"threads_started":1}
I20260808 00:46:39.104272 15488 tablet_service.cc:1467] Processing CreateTablet for tablet 8a71945254874cb9a896ba746ce7ad5b (DEFAULT_TABLE table=test-table [id=7a02c63512e44eca9959411775b44617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:39.104275 15697 tablet_service.cc:1467] Processing CreateTablet for tablet 8a71945254874cb9a896ba746ce7ad5b (DEFAULT_TABLE table=test-table [id=7a02c63512e44eca9959411775b44617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:39.104796 15697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a71945254874cb9a896ba746ce7ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:39.104796 15488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a71945254874cb9a896ba746ce7ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:39.108879 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.098195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40183 (local address 127.0.0.1:35294)
0808 00:46:39.099257 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.099271 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.099337 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.099544 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.099545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.099547 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.099594 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.099596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.100918 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.100919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.102048 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.102051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.103089 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.103090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.103146 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.103149 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.108660 (+ 5511us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.108763 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1040,"thread_start_us":126,"threads_started":1}
I20260808 00:46:39.108906 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.098578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35294 (local address 127.20.105.65:40183)
0808 00:46:39.098843 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.098847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.099350 (+ 503us) server_negotiation.cc:407] Connection header received
0808 00:46:39.099412 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.099415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.099432 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.099527 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.099666 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.100795 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.102139 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.102939 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.103003 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.103055 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.108716 (+ 5661us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.108797 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":116,"threads_started":1}
I20260808 00:46:39.109562 15815 tablet_bootstrap.cc:492] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: Bootstrap starting.
I20260808 00:46:39.110167 15281 tablet_service.cc:1467] Processing CreateTablet for tablet 8a71945254874cb9a896ba746ce7ad5b (DEFAULT_TABLE table=test-table [id=7a02c63512e44eca9959411775b44617]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:39.110474 15815 tablet_bootstrap.cc:654] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:39.110615 15281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a71945254874cb9a896ba746ce7ad5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:39.111285 15815 log.cc:824] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:39.111768 15814 tablet_bootstrap.cc:492] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: Bootstrap starting.
I20260808 00:46:39.112339 15815 tablet_bootstrap.cc:492] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: No bootstrap required, opened a new log
I20260808 00:46:39.112437 15815 ts_tablet_manager.cc:1397] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:46:39.112675 15814 tablet_bootstrap.cc:654] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:39.113305 15814 log.cc:824] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:39.114229 15814 tablet_bootstrap.cc:492] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: No bootstrap required, opened a new log
I20260808 00:46:39.114332 15814 ts_tablet_manager.cc:1397] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:46:39.114969 15815 raft_consensus.cc:348] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.115315 15820 tablet_bootstrap.cc:492] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab: Bootstrap starting.
I20260808 00:46:39.115381 15815 raft_consensus.cc:374] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:39.115444 15815 raft_consensus.cc:729] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44639a1d428648ce92a70da9c3cfed92, State: Initialized, Role: FOLLOWER
I20260808 00:46:39.115602 15815 consensus_queue.cc:260] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.115988 15788 heartbeater.cc:503] Master 127.20.105.126:33897 was elected leader, sending a full tablet report...
I20260808 00:46:39.115993 15815 ts_tablet_manager.cc:1428] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260808 00:46:39.116231 15820 tablet_bootstrap.cc:654] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:39.117046 15820 log.cc:824] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:39.117131 15814 raft_consensus.cc:348] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.117357 15814 raft_consensus.cc:374] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:39.117398 15814 raft_consensus.cc:729] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871b349e80054938a91df0c05f86cb05, State: Initialized, Role: FOLLOWER
I20260808 00:46:39.117561 15814 consensus_queue.cc:260] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.117949 15814 ts_tablet_manager.cc:1428] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:46:39.118031 15820 tablet_bootstrap.cc:492] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab: No bootstrap required, opened a new log
I20260808 00:46:39.118034 15596 heartbeater.cc:503] Master 127.20.105.126:33897 was elected leader, sending a full tablet report...
I20260808 00:46:39.118124 15820 ts_tablet_manager.cc:1397] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:46:39.120360 15820 raft_consensus.cc:348] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.120538 15820 raft_consensus.cc:374] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:39.120597 15820 raft_consensus.cc:729] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f17336e931b34947b9d02f34da87e3ab, State: Initialized, Role: FOLLOWER
I20260808 00:46:39.120750 15820 consensus_queue.cc:260] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.121110 15820 ts_tablet_manager.cc:1428] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 00:46:39.121176 15389 heartbeater.cc:503] Master 127.20.105.126:33897 was elected leader, sending a full tablet report...
I20260808 00:46:39.125690 15823 raft_consensus.cc:482] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:39.125810 15823 raft_consensus.cc:504] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.126237 15823 leader_election.cc:288] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f17336e931b34947b9d02f34da87e3ab (127.20.105.65:40183), 44639a1d428648ce92a70da9c3cfed92 (127.20.105.67:45841)
I20260808 00:46:39.126272 15821 raft_consensus.cc:482] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:39.126478 15821 raft_consensus.cc:504] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.127552 15821 leader_election.cc:288] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 871b349e80054938a91df0c05f86cb05 (127.20.105.66:39965), f17336e931b34947b9d02f34da87e3ab (127.20.105.65:40183)
I20260808 00:46:39.130861 15598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.126414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:45841 (local address 127.20.105.66:42827)
0808 00:46:39.126502 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.126517 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.126569 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.126900 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.126901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.126903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.127008 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.127011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.128108 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.128109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.129165 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.129167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.130578 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.130579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.130673 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.130677 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.130729 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.130777 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 00:46:39.130851 15792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.127037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39965 (local address 127.20.105.67:50241)
0808 00:46:39.127097 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.127108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.127144 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.127407 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.127408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.127410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.127488 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.127490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.128343 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.128344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.129745 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.129748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.130565 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.130566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.130644 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.130649 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.130704 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.130758 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 00:46:39.131100 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.126499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:42827 (local address 127.20.105.67:45841)
0808 00:46:39.126655 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.126657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.126666 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:46:39.126695 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.126696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.126707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.126757 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.127078 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.127998 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.129851 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.130473 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.130535 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.130544 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.130978 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.131039 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":130}
I20260808 00:46:39.131183 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.127231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:50241 (local address 127.20.105.66:39965)
0808 00:46:39.127276 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.127278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.127285 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:46:39.127308 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.127308 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.127319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.127367 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.127586 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.128241 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.129827 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.130423 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.130503 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.130529 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.131069 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.131108 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:46:39.131407 15727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a71945254874cb9a896ba746ce7ad5b" candidate_uuid: "871b349e80054938a91df0c05f86cb05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44639a1d428648ce92a70da9c3cfed92" is_pre_election: true
I20260808 00:46:39.131446 15527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a71945254874cb9a896ba746ce7ad5b" candidate_uuid: "44639a1d428648ce92a70da9c3cfed92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871b349e80054938a91df0c05f86cb05" is_pre_election: true
I20260808 00:46:39.131541 15727 raft_consensus.cc:2454] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 871b349e80054938a91df0c05f86cb05 in term 0.
I20260808 00:46:39.131552 15527 raft_consensus.cc:2454] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44639a1d428648ce92a70da9c3cfed92 in term 0.
I20260808 00:46:39.131831 15635 leader_election.cc:302] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44639a1d428648ce92a70da9c3cfed92, 871b349e80054938a91df0c05f86cb05; no voters:
I20260808 00:46:39.131829 15421 leader_election.cc:302] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44639a1d428648ce92a70da9c3cfed92, 871b349e80054938a91df0c05f86cb05; no voters:
I20260808 00:46:39.131984 15821 raft_consensus.cc:2781] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:46:39.132076 15821 raft_consensus.cc:482] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:46:39.132071 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.126718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:50581 (local address 127.20.105.65:40183)
0808 00:46:39.126768 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.126770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.126779 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:46:39.126809 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.126810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.126819 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.126883 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.127117 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.127998 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.130259 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.130828 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.130872 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.130880 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.131899 (+ 1019us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.131948 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:46:39.132107 15821 raft_consensus.cc:3037] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:39.132336 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.127229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40183 (local address 127.20.105.67:37385)
0808 00:46:39.128373 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.128383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.128450 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.128655 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.128655 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.128657 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.128735 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.128737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.129785 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.129786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.131073 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.131077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.132057 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.132058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.132137 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.132142 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.132196 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.132247 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":37,"thread_start_us":115,"threads_started":1}
I20260808 00:46:39.132407 15823 raft_consensus.cc:2781] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:46:39.132480 15823 raft_consensus.cc:482] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:46:39.132504 15823 raft_consensus.cc:3037] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:39.132666 15830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.127429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:37385 (local address 127.20.105.65:40183)
0808 00:46:39.127750 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.127751 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.128475 (+ 724us) server_negotiation.cc:407] Connection header received
0808 00:46:39.128508 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.128509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.128540 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.128644 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.128840 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.129686 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.131156 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.131928 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.131990 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.131997 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.132537 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.132585 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":97,"threads_started":1}
I20260808 00:46:39.132676 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.126433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40183 (local address 127.20.105.66:50581)
0808 00:46:39.126652 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.126665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.126734 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.126893 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.126894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.126897 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.127020 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.127022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.129254 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.129255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.130149 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.130152 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.131717 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.131718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.131794 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.131798 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.132339 (+ 541us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.132588 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":24,"thread_start_us":162,"threads_started":1}
I20260808 00:46:39.132946 15317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a71945254874cb9a896ba746ce7ad5b" candidate_uuid: "44639a1d428648ce92a70da9c3cfed92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f17336e931b34947b9d02f34da87e3ab" is_pre_election: true
I20260808 00:46:39.133136 15317 raft_consensus.cc:2454] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44639a1d428648ce92a70da9c3cfed92 in term 0.
I20260808 00:46:39.133351 15821 raft_consensus.cc:504] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.133536 15821 leader_election.cc:288] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [CANDIDATE]: Term 1 election: Requested vote from peers 871b349e80054938a91df0c05f86cb05 (127.20.105.66:39965), f17336e931b34947b9d02f34da87e3ab (127.20.105.65:40183)
I20260808 00:46:39.133607 15823 raft_consensus.cc:504] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.133659 15317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a71945254874cb9a896ba746ce7ad5b" candidate_uuid: "44639a1d428648ce92a70da9c3cfed92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f17336e931b34947b9d02f34da87e3ab"
I20260808 00:46:39.133754 15823 leader_election.cc:288] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [CANDIDATE]: Term 1 election: Requested vote from peers f17336e931b34947b9d02f34da87e3ab (127.20.105.65:40183), 44639a1d428648ce92a70da9c3cfed92 (127.20.105.67:45841)
I20260808 00:46:39.133744 15317 raft_consensus.cc:3037] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:39.133932 15727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a71945254874cb9a896ba746ce7ad5b" candidate_uuid: "871b349e80054938a91df0c05f86cb05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44639a1d428648ce92a70da9c3cfed92"
I20260808 00:46:39.133955 15527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a71945254874cb9a896ba746ce7ad5b" candidate_uuid: "44639a1d428648ce92a70da9c3cfed92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "871b349e80054938a91df0c05f86cb05"
I20260808 00:46:39.134033 15727 raft_consensus.cc:2379] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 871b349e80054938a91df0c05f86cb05 in current term 1: Already voted for candidate 44639a1d428648ce92a70da9c3cfed92 in this term.
I20260808 00:46:39.134054 15527 raft_consensus.cc:2379] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 44639a1d428648ce92a70da9c3cfed92 in current term 1: Already voted for candidate 871b349e80054938a91df0c05f86cb05 in this term.
I20260808 00:46:39.134232 15313 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a71945254874cb9a896ba746ce7ad5b" candidate_uuid: "871b349e80054938a91df0c05f86cb05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f17336e931b34947b9d02f34da87e3ab"
I20260808 00:46:39.134636 15423 leader_election.cc:302] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 871b349e80054938a91df0c05f86cb05; no voters: 44639a1d428648ce92a70da9c3cfed92, f17336e931b34947b9d02f34da87e3ab
I20260808 00:46:39.134758 15823 raft_consensus.cc:2726] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260808 00:46:39.134882 15317 raft_consensus.cc:2454] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44639a1d428648ce92a70da9c3cfed92 in term 1.
I20260808 00:46:39.133750 15314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a71945254874cb9a896ba746ce7ad5b" candidate_uuid: "871b349e80054938a91df0c05f86cb05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f17336e931b34947b9d02f34da87e3ab" is_pre_election: true
I20260808 00:46:39.135099 15314 raft_consensus.cc:2379] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 871b349e80054938a91df0c05f86cb05 in current term 1: Already voted for candidate 44639a1d428648ce92a70da9c3cfed92 in this term.
I20260808 00:46:39.135095 15634 leader_election.cc:302] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 44639a1d428648ce92a70da9c3cfed92, f17336e931b34947b9d02f34da87e3ab; no voters: 871b349e80054938a91df0c05f86cb05
I20260808 00:46:39.135191 15821 raft_consensus.cc:2781] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:39.135372 15821 raft_consensus.cc:686] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 1 LEADER]: Becoming Leader. State: Replica: 44639a1d428648ce92a70da9c3cfed92, State: Running, Role: LEADER
I20260808 00:46:39.135465 15821 consensus_queue.cc:237] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } }
I20260808 00:46:39.136605 15093 catalog_manager.cc:5495] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44639a1d428648ce92a70da9c3cfed92 (127.20.105.67). New cstate: current_term: 1 leader_uuid: "44639a1d428648ce92a70da9c3cfed92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44639a1d428648ce92a70da9c3cfed92" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45841 } health_report { overall_health: HEALTHY } } }
W20260808 00:46:39.138712 15391 tablet.cc:2369] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:46:39.145483 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.142520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37724 (local address 127.20.105.67:45841)
0808 00:46:39.142571 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.142573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.142590 (+ 17us) server_negotiation.cc:407] Connection header received
0808 00:46:39.142631 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.142632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.142662 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.142759 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:46:39.142964 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.143539 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.144546 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.144767 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.144853 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.145026 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:46:39.145153 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:46:39.145283 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.145340 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.145353 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:46:39.147826 15314 raft_consensus.cc:1261] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 1 FOLLOWER]: Refusing update from remote peer 44639a1d428648ce92a70da9c3cfed92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:39.147827 15527 raft_consensus.cc:1261] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 1 FOLLOWER]: Refusing update from remote peer 44639a1d428648ce92a70da9c3cfed92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:39.148223 15832 consensus_queue.cc:1035] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f17336e931b34947b9d02f34da87e3ab" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:39.148465 15821 consensus_queue.cc:1035] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "871b349e80054938a91df0c05f86cb05" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:39.151803 15835 mvcc.cc:205] Tried to move back new op lower bound from 7316070396505923584 to 7316070396459732992. Current Snapshot: MvccSnapshot[applied={T|T < 7316070396505923584}]
I20260808 00:46:39.152650 15836 mvcc.cc:205] Tried to move back new op lower bound from 7316070396505923584 to 7316070396459732992. Current Snapshot: MvccSnapshot[applied={T|T < 7316070396505923584}]
I20260808 00:46:39.155893 15092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:39.156010 15092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:39.157063 15092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d451021e2016454f9fd5334daf27fd3a: Sending DeleteTablet for 3 replicas of tablet 8a71945254874cb9a896ba746ce7ad5b
I20260808 00:46:39.157464 15281 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a71945254874cb9a896ba746ce7ad5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:39 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:35294
I20260808 00:46:39.157537 15697 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a71945254874cb9a896ba746ce7ad5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:39 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:37720
I20260808 00:46:39.157725 15849 tablet_replica.cc:330] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab: stopping tablet replica
I20260808 00:46:39.157788 15488 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a71945254874cb9a896ba746ce7ad5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:39 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:34440
I20260808 00:46:39.157848 15849 raft_consensus.cc:2229] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:39.157981 15849 raft_consensus.cc:2258] T 8a71945254874cb9a896ba746ce7ad5b P f17336e931b34947b9d02f34da87e3ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:39.158030 15851 tablet_replica.cc:330] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: stopping tablet replica
I20260808 00:46:39.158166 15850 tablet_replica.cc:330] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: stopping tablet replica
I20260808 00:46:39.158188 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 15201
I20260808 00:46:39.158272 15850 raft_consensus.cc:2229] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:46:39.158392 15850 raft_consensus.cc:2258] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:39.159132 15851 raft_consensus.cc:2229] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:39.159271 15851 raft_consensus.cc:2258] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:39.166668 15851 ts_tablet_manager.cc:1905] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:39.168874 15851 ts_tablet_manager.cc:1918] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:39.168977 15851 log.cc:1196] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/8a71945254874cb9a896ba746ce7ad5b
I20260808 00:46:39.169639 15851 ts_tablet_manager.cc:1939] T 8a71945254874cb9a896ba746ce7ad5b P 871b349e80054938a91df0c05f86cb05: Deleting consensus metadata
I20260808 00:46:39.170269 15071 catalog_manager.cc:4841] TS 871b349e80054938a91df0c05f86cb05 (127.20.105.66:39965): tablet 8a71945254874cb9a896ba746ce7ad5b (table test-table [id=7a02c63512e44eca9959411775b44617]) successfully deleted
I20260808 00:46:39.173426 15850 ts_tablet_manager.cc:1905] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:39.175796 15850 ts_tablet_manager.cc:1918] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:39.175899 15850 log.cc:1196] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/8a71945254874cb9a896ba746ce7ad5b
I20260808 00:46:39.176240 15850 ts_tablet_manager.cc:1939] T 8a71945254874cb9a896ba746ce7ad5b P 44639a1d428648ce92a70da9c3cfed92: Deleting consensus metadata
I20260808 00:46:39.176829 15068 catalog_manager.cc:4841] TS 44639a1d428648ce92a70da9c3cfed92 (127.20.105.67:45841): tablet 8a71945254874cb9a896ba746ce7ad5b (table test-table [id=7a02c63512e44eca9959411775b44617]) successfully deleted
W20260808 00:46:39.182052 15069 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:40183 (error 108)
I20260808 00:46:39.182389 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 15398
I20260808 00:46:39.198901 15807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.198602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40183 (local address 127.0.0.1:35300)
0808 00:46:39.198716 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.198820 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:40183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 00:46:39.199245 15069 catalog_manager.cc:4584] TS f17336e931b34947b9d02f34da87e3ab (127.20.105.65:40183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a71945254874cb9a896ba746ce7ad5b: Network error: Client connection negotiation failed: client connection to 127.20.105.65:40183: connect: Connection refused (error 111)
I20260808 00:46:39.204926 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 15602
I20260808 00:46:39.217989 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 15039
2026-08-08T00:46:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2428 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T00:46:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:39Z Disabled control of system clock
I20260808 00:46:39.263461 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44113
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:44113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:39.359894 15858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:39.360258 15858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:39.360299 15858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:39.361562 15858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:39.361640 15858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:39.361663 15858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:39.361686 15858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:39.361716 15858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:39.363251 15858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:44113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:44113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:39.363606 15858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:39.363998 15858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:39.369880 15867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.369913 15866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.369946 15869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:39.370419 15858 server_base.cc:1034] running on GCE node
I20260808 00:46:39.370779 15858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:39.371170 15858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:39.372323 15858 hybrid_clock.cc:648] HybridClock initialized: now 1786149999372297 us; error 53 us; skew 500 ppm
I20260808 00:46:39.373914 15858 webserver.cc:460] Webserver started at http://127.20.105.126:41373/ using document root <none> and password file <none>
I20260808 00:46:39.374289 15858 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:39.374377 15858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:39.374557 15858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:39.375885 15858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "6450acb9d81c4ef9baf065e197fefc7b"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "d184360520fd18b04f2294c0459002f4"
server_key_iv: "884f0c567547d15dedd842d8625d5166"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.376410 15858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "6450acb9d81c4ef9baf065e197fefc7b"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "d184360520fd18b04f2294c0459002f4"
server_key_iv: "884f0c567547d15dedd842d8625d5166"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.378088 15858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:46:39.379240 15875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.379511 15858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:39.379632 15858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "6450acb9d81c4ef9baf065e197fefc7b"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "d184360520fd18b04f2294c0459002f4"
server_key_iv: "884f0c567547d15dedd842d8625d5166"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.379715 15858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:39.392376 15858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:39.392803 15858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:39.392936 15858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:39.397887 15858 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:44113
I20260808 00:46:39.397981 15945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:44113 every 8 connection(s)
I20260808 00:46:39.398429 15858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:39.399828 15947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:39.399924 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 15858
I20260808 00:46:39.400141 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:39.400401 20901 external_mini_cluster.cc:1428] Setting key fbae1c2f0ad7329a6508beea6fba28de
I20260808 00:46:39.404311 15947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b: Bootstrap starting.
I20260808 00:46:39.405336 15947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:39.405805 15947 log.cc:824] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:39.406946 15947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b: No bootstrap required, opened a new log
I20260808 00:46:39.407987 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.401214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43118 (local address 127.20.105.126:44113)
0808 00:46:39.401452 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.401461 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.401492 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:46:39.402040 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.402049 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.402127 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.402275 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:39.402627 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.403970 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.405394 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.405994 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.406120 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.407159 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:39.407164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:39.407164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:39.407378 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:39.407426 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:39.407440 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:39.407523 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.407647 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.407755 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260808 00:46:39.409583 15947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6450acb9d81c4ef9baf065e197fefc7b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44113 } }
I20260808 00:46:39.409754 15947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:39.409777 15947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6450acb9d81c4ef9baf065e197fefc7b, State: Initialized, Role: FOLLOWER
I20260808 00:46:39.409914 15947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6450acb9d81c4ef9baf065e197fefc7b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44113 } }
I20260808 00:46:39.410002 15947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:39.410027 15947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:39.410060 15947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:39.410742 15947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6450acb9d81c4ef9baf065e197fefc7b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44113 } }
I20260808 00:46:39.410841 15947 leader_election.cc:302] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6450acb9d81c4ef9baf065e197fefc7b; no voters:
I20260808 00:46:39.411032 15947 leader_election.cc:288] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:39.411149 15954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:39.411373 15954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [term 1 LEADER]: Becoming Leader. State: Replica: 6450acb9d81c4ef9baf065e197fefc7b, State: Running, Role: LEADER
I20260808 00:46:39.411407 15947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:39.411482 15954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6450acb9d81c4ef9baf065e197fefc7b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44113 } }
I20260808 00:46:39.411981 15956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6450acb9d81c4ef9baf065e197fefc7b. Latest consensus state: current_term: 1 leader_uuid: "6450acb9d81c4ef9baf065e197fefc7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6450acb9d81c4ef9baf065e197fefc7b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44113 } } }
I20260808 00:46:39.412196 15956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:39.411986 15955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6450acb9d81c4ef9baf065e197fefc7b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6450acb9d81c4ef9baf065e197fefc7b" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 44113 } } }
I20260808 00:46:39.412467 15955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:39.412849 15967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:39.413497 15967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:39.415491 15967 catalog_manager.cc:1347] Generated new cluster ID: b3191ac31c914016ab32e99b4d495dff
I20260808 00:46:39.415556 15967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:39.426806 15967 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:39.427647 15967 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:39.434801 15967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b: Generated new TSK 0
I20260808 00:46:39.435101 15967 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:39.440477 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44113
--builtin_ntp_servers=127.20.105.84:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:39.564256 15979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:39.564548 15979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:39.564576 15979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:39.566018 15979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:39.566066 15979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:39.566098 15979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:46:39.567683 15979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:39.568073 15979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:39.568486 15979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:39.575364 15984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.575340 15985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.575340 15987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:39.575680 15979 server_base.cc:1034] running on GCE node
I20260808 00:46:39.575954 15979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:39.576365 15979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:39.577589 15979 hybrid_clock.cc:648] HybridClock initialized: now 1786149999577559 us; error 47 us; skew 500 ppm
I20260808 00:46:39.579406 15979 webserver.cc:460] Webserver started at http://127.20.105.65:34499/ using document root <none> and password file <none>
I20260808 00:46:39.579833 15979 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:39.579892 15979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:39.580248 15979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:39.581715 15979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "7413479500484478bb22b30a6a0feb35"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "e5af2e557c8d7e1209b19d3668431f00"
server_key_iv: "111ec7ad65ab04cd17764634bbd0d34a"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.582180 15979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "7413479500484478bb22b30a6a0feb35"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "e5af2e557c8d7e1209b19d3668431f00"
server_key_iv: "111ec7ad65ab04cd17764634bbd0d34a"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.584100 15979 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:46:39.585155 15998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.585316 15979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:39.585427 15979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "7413479500484478bb22b30a6a0feb35"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "e5af2e557c8d7e1209b19d3668431f00"
server_key_iv: "111ec7ad65ab04cd17764634bbd0d34a"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.585515 15979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:39.605756 15979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:39.606222 15979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:39.606405 15979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:39.606735 15979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:39.607182 15979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:39.607241 15979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.607288 15979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:39.607331 15979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.615717 15979 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:40061
I20260808 00:46:39.615767 16160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:40061 every 8 connection(s)
I20260808 00:46:39.616259 15979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:46:39.620524 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 15979
I20260808 00:46:39.620648 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:46:39.620877 20901 external_mini_cluster.cc:1428] Setting key cf85047f56a75438239bb71c4269352a
I20260808 00:46:39.622144 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44113
--builtin_ntp_servers=127.20.105.84:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 00:46:39.623019 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.617197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:54279 (local address 127.20.105.126:44113)
0808 00:46:39.617283 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.617286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.617595 (+ 309us) server_negotiation.cc:407] Connection header received
0808 00:46:39.618196 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.618197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.618215 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.618282 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:39.619151 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.619752 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.621146 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.621405 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.621491 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.622656 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:39.622658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:39.622658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:39.622664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:39.622687 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:39.622692 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:39.622768 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.622874 (+ 106us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.622917 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:46:39.623304 16165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.617201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44113 (local address 127.20.105.65:54279)
0808 00:46:39.617428 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.617479 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.618093 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.618360 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.618366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.618377 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:39.619011 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.619017 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.619869 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.619871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.621042 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.621046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.621507 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.621508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.621669 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.622475 (+ 806us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:39.622476 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:39.622525 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:39.622554 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:39.622769 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:39.622777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:39.622781 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.622876 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.623040 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260808 00:46:39.623962 16163 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44113
I20260808 00:46:39.624127 16163 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:39.624395 16163 heartbeater.cc:511] Master 127.20.105.126:44113 requested a full tablet report, sending...
I20260808 00:46:39.625155 15898 ts_manager.cc:194] Registered new tserver with Master: 7413479500484478bb22b30a6a0feb35 (127.20.105.65:40061)
I20260808 00:46:39.625897 15898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.65:54279
W20260808 00:46:39.723330 16167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:39.723619 16167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:39.723644 16167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:39.725045 16167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:39.725106 16167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:39.725136 16167 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:46:39.726629 16167 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:39.726946 16167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:39.727344 16167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:39.733706 16172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.733731 16177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:39.733812 16167 server_base.cc:1034] running on GCE node
W20260808 00:46:39.733731 16174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:39.734295 16167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:39.734651 16167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:39.735827 16167 hybrid_clock.cc:648] HybridClock initialized: now 1786149999735814 us; error 39 us; skew 500 ppm
I20260808 00:46:39.737648 16167 webserver.cc:460] Webserver started at http://127.20.105.66:44723/ using document root <none> and password file <none>
I20260808 00:46:39.737973 16167 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:39.738056 16167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:39.738245 16167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:39.739617 16167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "96783508cf31428f9cc1be77318210d5"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "4fa943c7424ae1d3c4caf1f07b222aea"
server_key_iv: "92ba0c3e4ba4800fb2e24a10f9a38a74"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.740156 16167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "96783508cf31428f9cc1be77318210d5"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "4fa943c7424ae1d3c4caf1f07b222aea"
server_key_iv: "92ba0c3e4ba4800fb2e24a10f9a38a74"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.742048 16167 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260808 00:46:39.742978 16187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.743191 16167 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:39.743299 16167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "96783508cf31428f9cc1be77318210d5"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "4fa943c7424ae1d3c4caf1f07b222aea"
server_key_iv: "92ba0c3e4ba4800fb2e24a10f9a38a74"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.743371 16167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:39.769927 16167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:39.770355 16167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:39.770494 16167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:39.770910 16167 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:39.771427 16167 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:39.771473 16167 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.771504 16167 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:39.771518 16167 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.779070 16167 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:46839
I20260808 00:46:39.779170 16349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:46839 every 8 connection(s)
I20260808 00:46:39.779631 16167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:46:39.781327 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 16167
I20260808 00:46:39.781430 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:46:39.781694 20901 external_mini_cluster.cc:1428] Setting key 658369ed6860cbf9eee0dbda510800c0
I20260808 00:46:39.783021 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44113
--builtin_ntp_servers=127.20.105.84:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 00:46:39.786985 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.780722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:38285 (local address 127.20.105.126:44113)
0808 00:46:39.780807 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.780810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.781108 (+ 298us) server_negotiation.cc:407] Connection header received
0808 00:46:39.781943 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.781945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.781962 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.782018 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:39.783102 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.783731 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.785069 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.785379 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.785468 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.786533 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:39.786535 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:39.786535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:39.786543 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:39.786571 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:39.786577 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:39.786643 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.786801 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.786838 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:46:39.787187 16352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.780709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44113 (local address 127.20.105.66:38285)
0808 00:46:39.780949 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.780993 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.781802 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.782140 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.782146 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.782158 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:39.782982 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.782990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.783877 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.783878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.784956 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.784959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.785501 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.785502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.785651 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.786360 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:39.786361 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:39.786399 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:39.786435 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:39.786690 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:39.786698 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:39.786702 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.786785 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.786903 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":108,"threads_started":1}
I20260808 00:46:39.787763 16350 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44113
I20260808 00:46:39.787902 16350 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:39.788188 16350 heartbeater.cc:511] Master 127.20.105.126:44113 requested a full tablet report, sending...
I20260808 00:46:39.788689 15898 ts_manager.cc:194] Registered new tserver with Master: 96783508cf31428f9cc1be77318210d5 (127.20.105.66:46839)
I20260808 00:46:39.789152 15898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.66:38285
W20260808 00:46:39.882530 16353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:39.882833 16353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:39.882870 16353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:39.884256 16353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:39.884325 16353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:39.884371 16353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:46:39.885847 16353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:44113
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:39.886176 16353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:39.886545 16353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:46:39.892899 16353 server_base.cc:1034] running on GCE node
W20260808 00:46:39.892864 16360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.892836 16361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:39.892871 16364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:39.893389 16353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:39.893754 16353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:39.894927 16353 hybrid_clock.cc:648] HybridClock initialized: now 1786149999894921 us; error 38 us; skew 500 ppm
I20260808 00:46:39.896682 16353 webserver.cc:460] Webserver started at http://127.20.105.67:33105/ using document root <none> and password file <none>
I20260808 00:46:39.896988 16353 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:39.897111 16353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:39.897294 16353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:39.898648 16353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "8656f05c15034d16a82ee1efccfe4dbe"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "e1866c4352c4e17fd3eb13eb3bdf4e9a"
server_key_iv: "cbc73b5234c8f36f22a4eaf5ba9dfe3d"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.899089 16353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "8656f05c15034d16a82ee1efccfe4dbe"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "e1866c4352c4e17fd3eb13eb3bdf4e9a"
server_key_iv: "cbc73b5234c8f36f22a4eaf5ba9dfe3d"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.900780 16353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:46:39.901726 16371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.901937 16353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:39.902060 16353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "8656f05c15034d16a82ee1efccfe4dbe"
format_stamp: "Formatted at 2026-08-08 00:46:39 on dist-test-slave-kvfs"
server_key: "e1866c4352c4e17fd3eb13eb3bdf4e9a"
server_key_iv: "cbc73b5234c8f36f22a4eaf5ba9dfe3d"
server_key_version: "encryptionkey@0"
I20260808 00:46:39.902172 16353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:39.911924 16353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:39.912422 16353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:39.912557 16353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:39.912804 16353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:39.913307 16353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:39.913376 16353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.913425 16353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:39.913452 16353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:39.920816 16353 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:42439
I20260808 00:46:39.920897 16541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:42439 every 8 connection(s)
I20260808 00:46:39.921361 16353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:46:39.922220 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 16353
I20260808 00:46:39.922346 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:46:39.922618 20901 external_mini_cluster.cc:1428] Setting key cbac466978eecb55f9c139c111f564b0
I20260808 00:46:39.929033 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.922489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:45767 (local address 127.20.105.126:44113)
0808 00:46:39.922596 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.922599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.922982 (+ 383us) server_negotiation.cc:407] Connection header received
0808 00:46:39.923939 (+ 957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.923941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.923960 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.924058 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:39.925042 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.925655 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.926937 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.927161 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.927233 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.928607 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:39.928609 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:39.928609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:39.928618 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:39.928642 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:39.928647 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:39.928712 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.928888 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.928934 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:46:39.929327 16546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.922444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:44113 (local address 127.20.105.67:45767)
0808 00:46:39.922716 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.922776 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.923764 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.924240 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.924247 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.924261 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:39.924916 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.924922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.925784 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.925785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.926754 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.926758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.927280 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.927282 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.927444 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.928375 (+ 931us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:39.928376 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:39.928421 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:39.928454 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:39.928746 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:39.928754 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:39.928758 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.928844 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.929007 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":119,"threads_started":1}
I20260808 00:46:39.930042 16542 heartbeater.cc:347] Connected to a master server at 127.20.105.126:44113
I20260808 00:46:39.930163 16542 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:39.930424 16542 heartbeater.cc:511] Master 127.20.105.126:44113 requested a full tablet report, sending...
I20260808 00:46:39.931003 15898 ts_manager.cc:194] Registered new tserver with Master: 8656f05c15034d16a82ee1efccfe4dbe (127.20.105.67:42439)
I20260808 00:46:39.931475 15898 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.67:45767
I20260808 00:46:39.935593 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:46:39.941119 15950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.937639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43126 (local address 127.20.105.126:44113)
0808 00:46:39.937737 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.937739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.937879 (+ 140us) server_negotiation.cc:407] Connection header received
0808 00:46:39.937914 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.937915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.937927 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.937977 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:39.938196 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.938778 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.939647 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.939859 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.939927 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.940793 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:39.940795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:39.940795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:39.940801 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:39.940818 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:39.940822 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:39.940917 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.941024 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.941037 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 00:46:39.943424 15898 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:46:39.945623 15898 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:46:39.954800 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.949424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40061 (local address 127.0.0.1:49288)
0808 00:46:39.949667 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.949722 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.949814 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.950489 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.950490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.950492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.950546 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.950548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.952103 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.952104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.953129 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.953132 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.954310 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.954312 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.954401 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.954416 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.954578 (+ 162us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.954661 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":122,"threads_started":1}
I20260808 00:46:39.954988 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.949739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:42439 (local address 127.0.0.1:58848)
0808 00:46:39.950643 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.950656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.950717 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.951118 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.951119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.951121 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.951171 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.951173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.952645 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.952646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.953677 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.953681 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.954726 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.954727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.954790 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.954795 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.954851 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.954901 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":880,"thread_start_us":106,"threads_started":1}
I20260808 00:46:39.955008 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.950406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58848 (local address 127.20.105.67:42439)
0808 00:46:39.950857 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.950862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.950872 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:46:39.950919 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.950921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.950951 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.951036 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.951249 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.952501 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.953793 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.954610 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.954712 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.954809 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.954871 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.954916 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":116,"threads_started":1}
I20260808 00:46:39.954800 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.949444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:46839 (local address 127.0.0.1:43776)
0808 00:46:39.949986 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:39.949997 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:39.950089 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:39.950329 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:39.950333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:39.950339 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:39.950440 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.950445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.951561 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.951562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.952965 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:39.952968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.954523 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.954525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:39.954605 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.954609 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:39.954670 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:46:39.954715 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"spinlock_wait_cycles":1408,"thread_start_us":146,"threads_started":1}
I20260808 00:46:39.955276 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.949692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.20.105.66:46839)
0808 00:46:39.949950 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.949958 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.950083 (+ 125us) server_negotiation.cc:407] Connection header received
0808 00:46:39.950188 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.950190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.950212 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.950320 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.950561 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.951465 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.953092 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.954196 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.954289 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.954416 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.955110 (+ 694us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.955175 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
I20260808 00:46:39.956156 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:39.949988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49288 (local address 127.20.105.65:40061)
0808 00:46:39.950223 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:39.950228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:39.950243 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:46:39.950299 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:39.950302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:39.950320 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:39.950474 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:39.950614 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.951957 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.953246 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:39.954192 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:39.954860 (+ 668us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:39.955928 (+ 1068us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:39.955963 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 00:46:39.956050 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":148,"threads_started":1}
I20260808 00:46:39.956193 16257 tablet_service.cc:1467] Processing CreateTablet for tablet 1d975cd99f7f4929968ed1197a039bd6 (DEFAULT_TABLE table=test-table [id=4d036dc21b14446290b1c10910d2e97d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:39.956324 16445 tablet_service.cc:1467] Processing CreateTablet for tablet 1d975cd99f7f4929968ed1197a039bd6 (DEFAULT_TABLE table=test-table [id=4d036dc21b14446290b1c10910d2e97d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:39.956781 16257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d975cd99f7f4929968ed1197a039bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:39.956781 16445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d975cd99f7f4929968ed1197a039bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:39.956895 16073 tablet_service.cc:1467] Processing CreateTablet for tablet 1d975cd99f7f4929968ed1197a039bd6 (DEFAULT_TABLE table=test-table [id=4d036dc21b14446290b1c10910d2e97d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:39.957262 16073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d975cd99f7f4929968ed1197a039bd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:39.960215 16567 tablet_bootstrap.cc:492] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Bootstrap starting.
I20260808 00:46:39.961042 16569 tablet_bootstrap.cc:492] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35: Bootstrap starting.
I20260808 00:46:39.961277 16570 tablet_bootstrap.cc:492] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Bootstrap starting.
I20260808 00:46:39.961714 16567 tablet_bootstrap.cc:654] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:39.961766 16569 tablet_bootstrap.cc:654] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:39.961973 16570 tablet_bootstrap.cc:654] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:39.962145 16567 log.cc:824] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:39.962145 16569 log.cc:824] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:39.962368 16570 log.cc:824] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:39.963122 16567 tablet_bootstrap.cc:492] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: No bootstrap required, opened a new log
I20260808 00:46:39.963122 16570 tablet_bootstrap.cc:492] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: No bootstrap required, opened a new log
I20260808 00:46:39.963194 16569 tablet_bootstrap.cc:492] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35: No bootstrap required, opened a new log
I20260808 00:46:39.963205 16570 ts_tablet_manager.cc:1397] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:46:39.963205 16567 ts_tablet_manager.cc:1397] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:46:39.963264 16569 ts_tablet_manager.cc:1397] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:46:39.965773 16569 raft_consensus.cc:348] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:39.965999 16569 raft_consensus.cc:374] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:39.966050 16569 raft_consensus.cc:729] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7413479500484478bb22b30a6a0feb35, State: Initialized, Role: FOLLOWER
I20260808 00:46:39.966203 16569 consensus_queue.cc:260] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:39.966148 16567 raft_consensus.cc:348] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:39.966143 16570 raft_consensus.cc:348] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:39.966363 16567 raft_consensus.cc:374] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:39.966363 16570 raft_consensus.cc:374] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:39.966403 16567 raft_consensus.cc:729] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96783508cf31428f9cc1be77318210d5, State: Initialized, Role: FOLLOWER
I20260808 00:46:39.966403 16570 raft_consensus.cc:729] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8656f05c15034d16a82ee1efccfe4dbe, State: Initialized, Role: FOLLOWER
I20260808 00:46:39.966548 16567 consensus_queue.cc:260] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:39.966548 16570 consensus_queue.cc:260] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:39.966621 16569 ts_tablet_manager.cc:1428] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260808 00:46:39.966815 16163 heartbeater.cc:503] Master 127.20.105.126:44113 was elected leader, sending a full tablet report...
I20260808 00:46:39.966856 16570 ts_tablet_manager.cc:1428] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:46:39.966856 16567 ts_tablet_manager.cc:1428] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 00:46:39.967012 16350 heartbeater.cc:503] Master 127.20.105.126:44113 was elected leader, sending a full tablet report...
I20260808 00:46:39.967173 16542 heartbeater.cc:503] Master 127.20.105.126:44113 was elected leader, sending a full tablet report...
W20260808 00:46:40.030439 16351 tablet.cc:2369] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:46:40.117278 16164 tablet.cc:2369] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:46:40.172222 16544 tablet.cc:2369] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:46:40.265477 16578 raft_consensus.cc:482] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:40.265683 16578 raft_consensus.cc:504] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:40.266237 16578 leader_election.cc:288] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96783508cf31428f9cc1be77318210d5 (127.20.105.66:46839), 8656f05c15034d16a82ee1efccfe4dbe (127.20.105.67:42439)
I20260808 00:46:40.271044 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:40.266596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:60885 (local address 127.20.105.66:46839)
0808 00:46:40.266701 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:40.266703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:40.266799 (+ 96us) server_negotiation.cc:407] Connection header received
0808 00:46:40.266859 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:40.266861 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:40.266876 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:40.266947 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:40.267210 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.268085 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.269890 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.270511 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.270605 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:40.270616 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:40.270888 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 00:46:40.270946 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 00:46:40.271044 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:40.266595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:42211 (local address 127.20.105.67:42439)
0808 00:46:40.266655 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:40.266657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:40.266801 (+ 144us) server_negotiation.cc:407] Connection header received
0808 00:46:40.266860 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:40.266860 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:40.266873 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:40.266947 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:40.267211 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.268090 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.269920 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.270512 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.270605 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:40.270616 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:40.270888 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 00:46:40.270947 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:46:40.271102 16584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:40.266508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:42439 (local address 127.20.105.65:42211)
0808 00:46:40.266698 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:40.266706 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:40.266792 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:40.266996 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:40.266996 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:40.266998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:40.267114 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:40.267117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.268237 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.268239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:40.269743 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:40.269748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.270647 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.270649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:40.270750 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:40.270758 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 00:46:40.270832 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:46:40.270902 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":86,"thread_start_us":118,"threads_started":1}
I20260808 00:46:40.271102 16585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:40.266486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:46839 (local address 127.20.105.65:60885)
0808 00:46:40.266681 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:40.266696 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:40.266792 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:40.266978 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:40.266979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:40.266982 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:40.267114 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:40.267117 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.268237 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.268239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:40.269743 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:40.269748 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.270667 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.270668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:40.270750 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:40.270756 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:46:40.270832 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 00:46:40.270902 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":107,"thread_start_us":134,"threads_started":1}
I20260808 00:46:40.271679 16474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d975cd99f7f4929968ed1197a039bd6" candidate_uuid: "7413479500484478bb22b30a6a0feb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8656f05c15034d16a82ee1efccfe4dbe" is_pre_election: true
I20260808 00:46:40.271785 16283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d975cd99f7f4929968ed1197a039bd6" candidate_uuid: "7413479500484478bb22b30a6a0feb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96783508cf31428f9cc1be77318210d5" is_pre_election: true
I20260808 00:46:40.271937 16474 raft_consensus.cc:2454] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7413479500484478bb22b30a6a0feb35 in term 0.
I20260808 00:46:40.272060 16283 raft_consensus.cc:2454] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7413479500484478bb22b30a6a0feb35 in term 0.
I20260808 00:46:40.272293 15999 leader_election.cc:302] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7413479500484478bb22b30a6a0feb35, 8656f05c15034d16a82ee1efccfe4dbe; no voters:
I20260808 00:46:40.272465 16578 raft_consensus.cc:2781] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:46:40.272532 16578 raft_consensus.cc:482] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:46:40.272565 16578 raft_consensus.cc:3037] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:40.273473 16578 raft_consensus.cc:504] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:40.273648 16578 leader_election.cc:288] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [CANDIDATE]: Term 1 election: Requested vote from peers 96783508cf31428f9cc1be77318210d5 (127.20.105.66:46839), 8656f05c15034d16a82ee1efccfe4dbe (127.20.105.67:42439)
I20260808 00:46:40.273810 16474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d975cd99f7f4929968ed1197a039bd6" candidate_uuid: "7413479500484478bb22b30a6a0feb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8656f05c15034d16a82ee1efccfe4dbe"
I20260808 00:46:40.273850 16283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d975cd99f7f4929968ed1197a039bd6" candidate_uuid: "7413479500484478bb22b30a6a0feb35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96783508cf31428f9cc1be77318210d5"
I20260808 00:46:40.273917 16474 raft_consensus.cc:3037] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:40.274000 16283 raft_consensus.cc:3037] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:40.274868 16474 raft_consensus.cc:2454] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7413479500484478bb22b30a6a0feb35 in term 1.
I20260808 00:46:40.274948 16283 raft_consensus.cc:2454] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7413479500484478bb22b30a6a0feb35 in term 1.
I20260808 00:46:40.275058 15999 leader_election.cc:302] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7413479500484478bb22b30a6a0feb35, 8656f05c15034d16a82ee1efccfe4dbe; no voters:
I20260808 00:46:40.275158 16578 raft_consensus.cc:2781] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:40.275429 16578 raft_consensus.cc:686] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [term 1 LEADER]: Becoming Leader. State: Replica: 7413479500484478bb22b30a6a0feb35, State: Running, Role: LEADER
I20260808 00:46:40.275568 16578 consensus_queue.cc:237] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } }
I20260808 00:46:40.276705 15895 catalog_manager.cc:5495] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7413479500484478bb22b30a6a0feb35 (127.20.105.65). New cstate: current_term: 1 leader_uuid: "7413479500484478bb22b30a6a0feb35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7413479500484478bb22b30a6a0feb35" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 40061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 } health_report { overall_health: UNKNOWN } } }
I20260808 00:46:40.312112 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:40.309146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49294 (local address 127.20.105.65:40061)
0808 00:46:40.309229 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:40.309233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:40.309244 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:46:40.309325 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:40.309326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:40.309354 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:40.309461 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:46:40.309634 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.310162 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.311063 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:40.311277 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:40.311358 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:40.311582 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:46:40.311698 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:46:40.311844 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:40.311884 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 00:46:40.311897 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:46:40.314883 16283 raft_consensus.cc:1261] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 1 FOLLOWER]: Refusing update from remote peer 7413479500484478bb22b30a6a0feb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:40.314883 16474 raft_consensus.cc:1261] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 1 FOLLOWER]: Refusing update from remote peer 7413479500484478bb22b30a6a0feb35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:40.315447 16586 consensus_queue.cc:1035] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96783508cf31428f9cc1be77318210d5" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 46839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:40.315654 16578 consensus_queue.cc:1035] T 1d975cd99f7f4929968ed1197a039bd6 P 7413479500484478bb22b30a6a0feb35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8656f05c15034d16a82ee1efccfe4dbe" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:40.322243 15895 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:40.322412 15895 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:40.323455 15895 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6450acb9d81c4ef9baf065e197fefc7b: Sending DeleteTablet for 3 replicas of tablet 1d975cd99f7f4929968ed1197a039bd6
I20260808 00:46:40.323854 16073 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d975cd99f7f4929968ed1197a039bd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:40 UTC) from {username='slave'} at 127.0.0.1:49288
I20260808 00:46:40.324064 16445 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d975cd99f7f4929968ed1197a039bd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:40 UTC) from {username='slave'} at 127.0.0.1:58848
I20260808 00:46:40.324273 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 15979
I20260808 00:46:40.324343 16257 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d975cd99f7f4929968ed1197a039bd6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:40 UTC) from {username='slave'} at 127.0.0.1:43776
I20260808 00:46:40.338001 16602 tablet_replica.cc:330] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: stopping tablet replica
I20260808 00:46:40.338271 16602 raft_consensus.cc:2229] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:40.338483 16602 raft_consensus.cc:2258] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:40.340292 16602 ts_tablet_manager.cc:1905] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:40.341670 16601 tablet_replica.cc:330] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: stopping tablet replica
I20260808 00:46:40.341893 16601 raft_consensus.cc:2229] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:40.342093 16601 raft_consensus.cc:2258] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:40.342705 16601 ts_tablet_manager.cc:1905] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:40.342803 16602 ts_tablet_manager.cc:1918] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:40.342900 16602 log.cc:1196] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/1d975cd99f7f4929968ed1197a039bd6
I20260808 00:46:40.343214 16602 ts_tablet_manager.cc:1939] T 1d975cd99f7f4929968ed1197a039bd6 P 96783508cf31428f9cc1be77318210d5: Deleting consensus metadata
I20260808 00:46:40.343822 15879 catalog_manager.cc:4841] TS 96783508cf31428f9cc1be77318210d5 (127.20.105.66:46839): tablet 1d975cd99f7f4929968ed1197a039bd6 (table test-table [id=4d036dc21b14446290b1c10910d2e97d]) successfully deleted
I20260808 00:46:40.344751 16601 ts_tablet_manager.cc:1918] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:40.344868 16601 log.cc:1196] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/1d975cd99f7f4929968ed1197a039bd6
I20260808 00:46:40.345180 16601 ts_tablet_manager.cc:1939] T 1d975cd99f7f4929968ed1197a039bd6 P 8656f05c15034d16a82ee1efccfe4dbe: Deleting consensus metadata
I20260808 00:46:40.345716 15876 catalog_manager.cc:4841] TS 8656f05c15034d16a82ee1efccfe4dbe (127.20.105.67:42439): tablet 1d975cd99f7f4929968ed1197a039bd6 (table test-table [id=4d036dc21b14446290b1c10910d2e97d]) successfully deleted
W20260808 00:46:40.348340 15876 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:40061 (error 108)
I20260808 00:46:40.349100 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:40.348888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:40061 (local address 127.0.0.1:49300)
0808 00:46:40.348962 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:40.349014 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:40061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 00:46:40.349174 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 16167
W20260808 00:46:40.349280 15876 catalog_manager.cc:4584] TS 7413479500484478bb22b30a6a0feb35 (127.20.105.65:40061): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1d975cd99f7f4929968ed1197a039bd6: Network error: Client connection negotiation failed: client connection to 127.20.105.65:40061: connect: Connection refused (error 111)
I20260808 00:46:40.372344 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 16353
I20260808 00:46:40.386703 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 15858
2026-08-08T00:46:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1163 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:40 dist-test-slave-kvfs krb5kdc[16607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:40 dist-test-slave-kvfs krb5kdc[16607](info): set up 2 sockets
Aug 08 00:46:40 dist-test-slave-kvfs krb5kdc[16607](info): commencing operation
krb5kdc: starting...
W20260808 00:46:41.694377 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:41 dist-test-slave-kvfs krb5kdc[16607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:46:41.864971 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:46611
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:33655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:46611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:41.974105 16629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:41.974400 16629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:41.974427 16629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:41.975749 16629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:41.975798 16629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:41.975813 16629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:41.975827 16629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:41.975840 16629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:41.977711 16629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:33655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:46611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:46611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:41.978057 16629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:41.978391 16629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:46:41.984701 16629 server_base.cc:1034] running on GCE node
W20260808 00:46:41.984642 16635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:41.984647 16636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:41.984647 16641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:41.985397 16629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:41.985813 16629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:41.987010 16629 hybrid_clock.cc:648] HybridClock initialized: now 1786150001986989 us; error 53 us; skew 500 ppm
Aug 08 00:46:41 dist-test-slave-kvfs krb5kdc[16607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150001, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:41.997654 16629 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:46:41.998255 16629 webserver.cc:460] Webserver started at http://127.20.105.126:41739/ using document root <none> and password file <none>
I20260808 00:46:41.998564 16629 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:41.998620 16629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:41.998792 16629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:42.000231 16629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "25a64cac54f94abca59f0715b92b8c4c"
format_stamp: "Formatted at 2026-08-08 00:46:41 on dist-test-slave-kvfs"
server_key: "059588601fe503fcae00b6ac089660ec"
server_key_iv: "543726947be598fac388539dd8931497"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.000695 16629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "25a64cac54f94abca59f0715b92b8c4c"
format_stamp: "Formatted at 2026-08-08 00:46:41 on dist-test-slave-kvfs"
server_key: "059588601fe503fcae00b6ac089660ec"
server_key_iv: "543726947be598fac388539dd8931497"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.002379 16629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:46:42.003345 16650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.003615 16629 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:46:42.003765 16629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "25a64cac54f94abca59f0715b92b8c4c"
format_stamp: "Formatted at 2026-08-08 00:46:41 on dist-test-slave-kvfs"
server_key: "059588601fe503fcae00b6ac089660ec"
server_key_iv: "543726947be598fac388539dd8931497"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.003866 16629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:42.025413 16629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:42.027848 16629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:42.028110 16629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:42.033711 16629 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:46611
I20260808 00:46:42.033744 16725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:46611 every 8 connection(s)
I20260808 00:46:42.034230 16629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:42.035410 16726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:42.036448 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 16629
I20260808 00:46:42.036587 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:42.036847 20901 external_mini_cluster.cc:1428] Setting key 2fbfa24a35cf29d6842a9c8622bc4ac6
I20260808 00:46:42.039068 16726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c: Bootstrap starting.
I20260808 00:46:42.039886 16726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:42.040284 16726 log.cc:824] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:42.041136 16726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c: No bootstrap required, opened a new log
I20260808 00:46:42.043952 16726 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a64cac54f94abca59f0715b92b8c4c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46611 } }
I20260808 00:46:42.044219 16726 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:42.044279 16726 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25a64cac54f94abca59f0715b92b8c4c, State: Initialized, Role: FOLLOWER
I20260808 00:46:42.044441 16726 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a64cac54f94abca59f0715b92b8c4c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46611 } }
Aug 08 00:46:42 dist-test-slave-kvfs krb5kdc[16607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:42.044544 16726 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:42.044597 16726 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:42.044656 16726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:42.045502 16726 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a64cac54f94abca59f0715b92b8c4c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46611 } }
I20260808 00:46:42.045660 16726 leader_election.cc:302] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25a64cac54f94abca59f0715b92b8c4c; no voters:
I20260808 00:46:42.045883 16726 leader_election.cc:288] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:42.045966 16732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:42.046198 16732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [term 1 LEADER]: Becoming Leader. State: Replica: 25a64cac54f94abca59f0715b92b8c4c, State: Running, Role: LEADER
I20260808 00:46:42.046267 16726 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:42.046326 16732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a64cac54f94abca59f0715b92b8c4c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46611 } }
I20260808 00:46:42.046844 16734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25a64cac54f94abca59f0715b92b8c4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a64cac54f94abca59f0715b92b8c4c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46611 } } }
I20260808 00:46:42.046845 16733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25a64cac54f94abca59f0715b92b8c4c. Latest consensus state: current_term: 1 leader_uuid: "25a64cac54f94abca59f0715b92b8c4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25a64cac54f94abca59f0715b92b8c4c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 46611 } } }
I20260808 00:46:42.047057 16734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:42.047071 16733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:42.047451 16743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:42.048146 16743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:42.050051 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.037543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59066 (local address 127.20.105.126:46611)
0808 00:46:42.037773 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.037782 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.037813 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:46:42.038293 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.038300 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.038353 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.038542 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:42.039344 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.040498 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.041355 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.042010 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.042153 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.045094 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:42.045099 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:42.045099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:42.045115 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:42.048325 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.048818 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.048821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.048821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.048915 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.049147 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.049148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.049148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.049397 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:42.049605 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.049686 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.049844 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260808 00:46:42.050832 16743 catalog_manager.cc:1347] Generated new cluster ID: a18cdf8f5b564ef08aacc549bd86f3ef
I20260808 00:46:42.050922 16743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:42.075145 16743 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:42.075840 16743 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:42.087401 16743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c: Generated new TSK 0
I20260808 00:46:42.087744 16743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.105.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
Entry for principal kudu/127.20.105.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab.
I20260808 00:46:42.135900 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46611
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:33655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:42.240953 16759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:42.241348 16759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:42.241385 16759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:42.242676 16759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:42.242726 16759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:42.242755 16759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:46:42.244503 16759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:33655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.65.keytab
--principal=kudu/127.20.105.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16759
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:42.244776 16759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:42.245015 16759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:42.251900 16767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:42.251888 16766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:42.251907 16771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:42.252415 16759 server_base.cc:1034] running on GCE node
I20260808 00:46:42.252668 16759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:42.252972 16759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:42.254148 16759 hybrid_clock.cc:648] HybridClock initialized: now 1786150002254125 us; error 32 us; skew 500 ppm
Aug 08 00:46:42 dist-test-slave-kvfs krb5kdc[16607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:42.257467 16759 init.cc:377] Logged in from keytab as kudu/127.20.105.65@KRBTEST.COM (short username kudu)
I20260808 00:46:42.258050 16759 webserver.cc:460] Webserver started at http://127.20.105.65:35569/ using document root <none> and password file <none>
I20260808 00:46:42.258337 16759 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:42.258390 16759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:42.258543 16759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:42.259902 16759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "016a6ca30eef43c4885c20c45d5ff3c3"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "5585a148a960634141686752908d15e1"
server_key_iv: "2af44415d2644755dced16ab3e4c9d51"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.260412 16759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "016a6ca30eef43c4885c20c45d5ff3c3"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "5585a148a960634141686752908d15e1"
server_key_iv: "2af44415d2644755dced16ab3e4c9d51"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.262115 16759 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:46:42.263110 16780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.263350 16759 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:42.263465 16759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "016a6ca30eef43c4885c20c45d5ff3c3"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "5585a148a960634141686752908d15e1"
server_key_iv: "2af44415d2644755dced16ab3e4c9d51"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.263554 16759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:42.271289 16759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:42.273684 16759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:42.273860 16759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:42.274176 16759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:42.274606 16759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:42.274668 16759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.274708 16759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:42.274736 16759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.282248 16759 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:38517
I20260808 00:46:42.282307 16941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:38517 every 8 connection(s)
I20260808 00:46:42.282728 16759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:46:42.284974 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 16759
I20260808 00:46:42.285156 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:46:42.285413 20901 external_mini_cluster.cc:1428] Setting key 7faf8b62834a496b6b424d78baa73fcb
Aug 08 00:46:42 dist-test-slave-kvfs krb5kdc[16607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.65@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:42.296556 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.283737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:50725 (local address 127.20.105.126:46611)
0808 00:46:42.283842 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.283844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.284198 (+ 354us) server_negotiation.cc:407] Connection header received
0808 00:46:42.284878 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.284880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.284895 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.284961 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:42.286419 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.287260 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.288571 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.288937 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.289030 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.292457 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:42.292459 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:42.292460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:42.292474 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:42.294712 (+ 2238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.295239 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.295240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.295241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.295324 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.295633 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.295634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.295634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.295814 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:42.295895 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.296296 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.296416 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 00:46:42.296679 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.283708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:46611 (local address 127.20.105.65:50725)
0808 00:46:42.283994 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.284088 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:42.284751 (+ 663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:42.285049 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:42.285055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:42.285977 (+ 922us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:42.286251 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.286259 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.287389 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.287390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.288478 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.288482 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.289160 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.289162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.289370 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.290111 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:42.290126 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:42.292334 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:42.294822 (+ 2488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:42.294832 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:42.294834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:42.295135 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:42.295401 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:42.295402 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:42.295403 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:42.295544 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:42.295884 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:42.295890 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:42.296129 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 00:46:42.296229 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 00:46:42.296400 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":116,"threads_started":1}
I20260808 00:46:42.297355 16942 heartbeater.cc:347] Connected to a master server at 127.20.105.126:46611
I20260808 00:46:42.297497 16942 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:42.297741 16942 heartbeater.cc:511] Master 127.20.105.126:46611 requested a full tablet report, sending...
I20260808 00:46:42.298590 16671 ts_manager.cc:194] Registered new tserver with Master: 016a6ca30eef43c4885c20c45d5ff3c3 (127.20.105.65:38517)
WARNING: no policy specified for kudu/127.20.105.66@KRBTEST.COM; defaulting to no policy
I20260808 00:46:42.299510 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.65@KRBTEST.COM'} at 127.20.105.65:50725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
Entry for principal kudu/127.20.105.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab.
I20260808 00:46:42.324944 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46611
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:33655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:42.424793 16947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:42.425084 16947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:42.425113 16947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:42.426465 16947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:42.426524 16947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:42.426556 16947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:46:42.428342 16947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:33655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.66.keytab
--principal=kudu/127.20.105.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:42.428704 16947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:42.429032 16947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:42.435880 16958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:42.435930 16957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:42.435880 16961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:42.436528 16947 server_base.cc:1034] running on GCE node
I20260808 00:46:42.436766 16947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:42.437040 16947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:42.438205 16947 hybrid_clock.cc:648] HybridClock initialized: now 1786150002438184 us; error 32 us; skew 500 ppm
Aug 08 00:46:42 dist-test-slave-kvfs krb5kdc[16607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:42.441751 16947 init.cc:377] Logged in from keytab as kudu/127.20.105.66@KRBTEST.COM (short username kudu)
I20260808 00:46:42.442390 16947 webserver.cc:460] Webserver started at http://127.20.105.66:36007/ using document root <none> and password file <none>
I20260808 00:46:42.442827 16947 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:42.442925 16947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:42.443125 16947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:42.444656 16947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "747e9a6ba6c9be82c1ebc6610ecc1e30"
server_key_iv: "41523cb56610f125e98bca318bcd8ef3"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.445130 16947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "747e9a6ba6c9be82c1ebc6610ecc1e30"
server_key_iv: "41523cb56610f125e98bca318bcd8ef3"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.446946 16947 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:46:42.448082 16975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.448343 16947 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:42.448465 16947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "747e9a6ba6c9be82c1ebc6610ecc1e30"
server_key_iv: "41523cb56610f125e98bca318bcd8ef3"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.448539 16947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:42.462431 16947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:42.465134 16947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:42.465283 16947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:42.465575 16947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:42.466004 16947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:42.466045 16947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.466101 16947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:42.466130 16947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.473238 16947 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:42739
I20260808 00:46:42.473340 17124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:42739 every 8 connection(s)
I20260808 00:46:42.473765 16947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:46:42 dist-test-slave-kvfs krb5kdc[16607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.66@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:42.483657 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 16947
I20260808 00:46:42.483814 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:46:42.484128 20901 external_mini_cluster.cc:1428] Setting key 5e54b0418ce394a8ebc1ec4b24e6341a
I20260808 00:46:42.484943 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.474733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:51175 (local address 127.20.105.126:46611)
0808 00:46:42.474936 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.474939 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.475195 (+ 256us) server_negotiation.cc:407] Connection header received
0808 00:46:42.475851 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.475852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.475868 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.475936 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:42.477170 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.477761 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.478896 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.479112 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.479196 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.481748 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:42.481750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:42.481751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:42.481764 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:42.483079 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.483612 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.483613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.483614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.483684 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.484071 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.484072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.484072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.484268 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:42.484332 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.484718 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.484827 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:46:42.485147 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.474676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:46611 (local address 127.20.105.66:51175)
0808 00:46:42.475016 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.475080 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:42.475720 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:42.476095 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:42.476099 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:42.476787 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:42.477049 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.477055 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.477875 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.477876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.478789 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.478792 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.479203 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.479204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.479356 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.479956 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:42.479965 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:42.481633 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:42.483169 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:42.483178 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:42.483181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:42.483520 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:42.483788 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:42.483790 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:42.483790 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:42.483948 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:42.484326 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:42.484333 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:42.484535 (+ 202us) client_negotiation.cc:769] Sending connection context
0808 00:46:42.484671 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 00:46:42.484828 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":116,"threads_started":1}
I20260808 00:46:42.485731 17126 heartbeater.cc:347] Connected to a master server at 127.20.105.126:46611
I20260808 00:46:42.485852 17126 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:42.486095 17126 heartbeater.cc:511] Master 127.20.105.126:46611 requested a full tablet report, sending...
I20260808 00:46:42.486603 16671 ts_manager.cc:194] Registered new tserver with Master: 0ab9fac3247b4b0e9ec932a1a8cd7d04 (127.20.105.66:42739)
I20260808 00:46:42.487033 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.66@KRBTEST.COM'} at 127.20.105.66:51175
WARNING: no policy specified for kudu/127.20.105.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
Entry for principal kudu/127.20.105.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab.
I20260808 00:46:42.520726 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46611
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:33655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:42.622323 17134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:42.622624 17134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:42.622670 17134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:42.623907 17134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:42.623982 17134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:42.624028 17134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:46:42.625713 17134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:33655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.67.keytab
--principal=kudu/127.20.105.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:42.626060 17134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:42.626377 17134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:42.632531 17143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:42.632537 17150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:42.632709 17142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:42.632871 17134 server_base.cc:1034] running on GCE node
I20260808 00:46:42.633176 17134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:42.633491 17134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:42.634704 17134 hybrid_clock.cc:648] HybridClock initialized: now 1786150002634692 us; error 41 us; skew 500 ppm
Aug 08 00:46:42 dist-test-slave-kvfs krb5kdc[16607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:42.638193 17134 init.cc:377] Logged in from keytab as kudu/127.20.105.67@KRBTEST.COM (short username kudu)
I20260808 00:46:42.638854 17134 webserver.cc:460] Webserver started at http://127.20.105.67:35327/ using document root <none> and password file <none>
I20260808 00:46:42.639241 17134 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:42.639328 17134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:42.639616 17134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:42.641134 17134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "c99c146127fa45a59fe11d6f710a69dd"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "024feb4fade00d3ca36adc4da4a18d21"
server_key_iv: "345c9efa996d839419b2a0e73e4018d5"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.641598 17134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "c99c146127fa45a59fe11d6f710a69dd"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "024feb4fade00d3ca36adc4da4a18d21"
server_key_iv: "345c9efa996d839419b2a0e73e4018d5"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.643326 17134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:46:42.644424 17158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.644749 17134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:42.644901 17134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "c99c146127fa45a59fe11d6f710a69dd"
format_stamp: "Formatted at 2026-08-08 00:46:42 on dist-test-slave-kvfs"
server_key: "024feb4fade00d3ca36adc4da4a18d21"
server_key_iv: "345c9efa996d839419b2a0e73e4018d5"
server_key_version: "encryptionkey@0"
I20260808 00:46:42.645000 17134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:42.655371 17134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:42.658154 17134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:42.658370 17134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:42.658744 17134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:42.659176 17134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:42.659243 17134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.659296 17134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:42.659327 17134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:42.667697 17134 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:41423
I20260808 00:46:42.667812 17311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:41423 every 8 connection(s)
I20260808 00:46:42.668305 17134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:46:42.669752 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 17134
I20260808 00:46:42.669896 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:46:42.670135 20901 external_mini_cluster.cc:1428] Setting key 2865c16587ca27168940f6678e8ba70b
Aug 08 00:46:42 dist-test-slave-kvfs krb5kdc[16607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150002, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.67@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:46:42.680457 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.669387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:40971 (local address 127.20.105.126:46611)
0808 00:46:42.669465 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.669468 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.669756 (+ 288us) server_negotiation.cc:407] Connection header received
0808 00:46:42.670573 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.670575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.670591 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.670660 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:42.671900 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.672597 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.673818 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.674058 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.674149 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.677005 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:42.677007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:42.677010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:42.677024 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:42.678651 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.679186 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.679188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.679188 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.679264 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.679565 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.679566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.679566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.679722 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:42.679815 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.680225 (+ 410us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.680331 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:46:42.680730 17314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.669342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:46611 (local address 127.20.105.67:40971)
0808 00:46:42.669593 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.669646 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:42.670449 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:42.670759 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:42.670763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:42.671510 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:42.671773 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.671779 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.672709 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.672710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.673699 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.673702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.674155 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.674157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.674313 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.675011 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:46:42.675021 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:42.676809 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:42.678785 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:42.678794 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:42.678798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:42.679093 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:42.679354 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:46:42.679355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:46:42.679355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:46:42.679486 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:46:42.679802 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:42.679811 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:42.679989 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 00:46:42.680188 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 00:46:42.680409 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":109,"threads_started":1}
I20260808 00:46:42.681505 17312 heartbeater.cc:347] Connected to a master server at 127.20.105.126:46611
I20260808 00:46:42.681676 17312 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:42.681928 17312 heartbeater.cc:511] Master 127.20.105.126:46611 requested a full tablet report, sending...
I20260808 00:46:42.682442 16671 ts_manager.cc:194] Registered new tserver with Master: c99c146127fa45a59fe11d6f710a69dd (127.20.105.67:41423)
I20260808 00:46:42.682909 16671 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.105.67@KRBTEST.COM'} at 127.20.105.67:40971
I20260808 00:46:42.693570 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:46:42.703332 16730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.695512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59070 (local address 127.20.105.126:46611)
0808 00:46:42.695608 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.695610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.695707 (+ 97us) server_negotiation.cc:407] Connection header received
0808 00:46:42.695812 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.695813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.695826 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.695893 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:42.696644 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.697199 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.697975 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.698425 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.698575 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.700153 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:42.700154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:42.700155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:42.700162 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:42.701739 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.702198 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.702200 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.702201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.702277 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:42.702532 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:42.702533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:42.702533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:42.702747 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:42.702900 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.703063 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.703184 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 00:46:42.705600 16671 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:46:42.707798 16671 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:46:42.717190 17335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.711080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:38517 (local address 127.0.0.1:33302)
0808 00:46:42.711618 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.711674 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:42.711766 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:42.712310 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:42.712311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:42.712313 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:42.712372 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.712374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.714045 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.714046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.715287 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.715290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.716739 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.716740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.716816 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.716830 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 00:46:42.716948 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 00:46:42.717023 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":86,"thread_start_us":82,"threads_started":1}
I20260808 00:46:42.717176 17338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.711269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33302 (local address 127.20.105.65:38517)
0808 00:46:42.712123 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.712128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.712141 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:46:42.712192 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.712194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.712212 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.712299 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:42.712578 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.713897 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.715399 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.716530 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.716606 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.716694 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.716975 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.717030 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":142,"threads_started":1}
I20260808 00:46:42.717412 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.711320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59484 (local address 127.20.105.66:42739)
0808 00:46:42.711607 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.711613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.711763 (+ 150us) server_negotiation.cc:407] Connection header received
0808 00:46:42.711995 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.711999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.712045 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.712153 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:42.712509 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.713916 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.715768 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.716690 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.716771 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.716830 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.717235 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.717293 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":122,"threads_started":1}
I20260808 00:46:42.717191 17341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.711080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:42739 (local address 127.0.0.1:59484)
0808 00:46:42.711659 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.711674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:42.711766 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:42.712174 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:42.712178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:42.712180 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:42.712282 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.712284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.714064 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.714066 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.715648 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.715652 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.716869 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.716870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.716957 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.716961 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:42.717013 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:46:42.717062 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":2,"thread_start_us":124,"threads_started":1}
I20260808 00:46:42.718863 17333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.711364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:41423 (local address 127.0.0.1:52638)
0808 00:46:42.711876 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.711884 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:42.711931 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:42.714384 (+ 2453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:42.714385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:42.714388 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:42.714463 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.714464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.716012 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.716014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.717245 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.717248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.718575 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.718576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.718629 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.718634 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:42.718717 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:46:42.718771 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":21,"thread_start_us":86,"threads_started":1}
I20260808 00:46:42.718894 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.711524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.20.105.67:41423)
0808 00:46:42.714162 (+ 2638us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.714167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.714180 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:46:42.714232 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.714234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.714255 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.714356 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:42.714602 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.715883 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.717478 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.718378 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.718475 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.718540 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.718740 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.718796 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2574,"thread_start_us":120,"threads_started":1}
I20260808 00:46:42.718984 16847 tablet_service.cc:1467] Processing CreateTablet for tablet b96a23cfc00641598ecd0913ea850a3a (DEFAULT_TABLE table=test-table [id=8d3b929df1c84f989a02d415f975f178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:42.718983 17035 tablet_service.cc:1467] Processing CreateTablet for tablet b96a23cfc00641598ecd0913ea850a3a (DEFAULT_TABLE table=test-table [id=8d3b929df1c84f989a02d415f975f178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:42.719498 16847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b96a23cfc00641598ecd0913ea850a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:42.719498 17035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b96a23cfc00641598ecd0913ea850a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:42.720916 17221 tablet_service.cc:1467] Processing CreateTablet for tablet b96a23cfc00641598ecd0913ea850a3a (DEFAULT_TABLE table=test-table [id=8d3b929df1c84f989a02d415f975f178]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:42.721344 17221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b96a23cfc00641598ecd0913ea850a3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:42.724227 17344 tablet_bootstrap.cc:492] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: Bootstrap starting.
I20260808 00:46:42.724484 17349 tablet_bootstrap.cc:492] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3: Bootstrap starting.
I20260808 00:46:42.725229 17349 tablet_bootstrap.cc:654] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:42.725236 17344 tablet_bootstrap.cc:654] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:42.725476 17350 tablet_bootstrap.cc:492] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: Bootstrap starting.
I20260808 00:46:42.725986 17344 log.cc:824] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:42.726148 17349 log.cc:824] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:42.726460 17350 tablet_bootstrap.cc:654] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:42.727092 17344 tablet_bootstrap.cc:492] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: No bootstrap required, opened a new log
I20260808 00:46:42.727145 17349 tablet_bootstrap.cc:492] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3: No bootstrap required, opened a new log
I20260808 00:46:42.727164 17350 log.cc:824] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:42.727197 17344 ts_tablet_manager.cc:1397] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:46:42.727218 17349 ts_tablet_manager.cc:1397] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:46:42.728200 17350 tablet_bootstrap.cc:492] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: No bootstrap required, opened a new log
I20260808 00:46:42.728288 17350 ts_tablet_manager.cc:1397] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:46:42.729527 17344 raft_consensus.cc:348] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.729751 17344 raft_consensus.cc:374] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:42.729811 17344 raft_consensus.cc:729] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ab9fac3247b4b0e9ec932a1a8cd7d04, State: Initialized, Role: FOLLOWER
I20260808 00:46:42.729681 17349 raft_consensus.cc:348] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.729903 17349 raft_consensus.cc:374] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:42.729959 17349 raft_consensus.cc:729] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016a6ca30eef43c4885c20c45d5ff3c3, State: Initialized, Role: FOLLOWER
I20260808 00:46:42.729992 17344 consensus_queue.cc:260] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.730113 17349 consensus_queue.cc:260] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.730396 17344 ts_tablet_manager.cc:1428] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260808 00:46:42.730448 17349 ts_tablet_manager.cc:1428] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:46:42.730621 16942 heartbeater.cc:503] Master 127.20.105.126:46611 was elected leader, sending a full tablet report...
I20260808 00:46:42.730737 17350 raft_consensus.cc:348] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.730947 17350 raft_consensus.cc:374] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:42.730996 17350 raft_consensus.cc:729] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c99c146127fa45a59fe11d6f710a69dd, State: Initialized, Role: FOLLOWER
I20260808 00:46:42.731004 17126 heartbeater.cc:503] Master 127.20.105.126:46611 was elected leader, sending a full tablet report...
I20260808 00:46:42.731119 17350 consensus_queue.cc:260] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.731456 17350 ts_tablet_manager.cc:1428] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:46:42.731498 17312 heartbeater.cc:503] Master 127.20.105.126:46611 was elected leader, sending a full tablet report...
W20260808 00:46:42.783808 16943 tablet.cc:2369] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:46:42.792884 17359 raft_consensus.cc:482] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:42.793066 17359 raft_consensus.cc:504] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.793619 17359 leader_election.cc:288] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ab9fac3247b4b0e9ec932a1a8cd7d04 (127.20.105.66:42739), 016a6ca30eef43c4885c20c45d5ff3c3 (127.20.105.65:38517)
I20260808 00:46:42.797158 17360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.793895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:38517 (local address 127.20.105.67:54705)
0808 00:46:42.794203 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.794216 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:42.794318 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:42.794476 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:42.794477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:42.794478 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:42.794565 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.794569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.795316 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.795317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.796211 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.796213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.796821 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.796822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.796919 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.796923 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:42.797002 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:46:42.797047 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"mutex_wait_us":19,"thread_start_us":189,"threads_started":1}
I20260808 00:46:42.797139 17338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.794046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:54705 (local address 127.20.105.65:38517)
0808 00:46:42.794102 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.794104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.794301 (+ 197us) server_negotiation.cc:407] Connection header received
0808 00:46:42.794385 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.794386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.794397 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.794474 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:42.794650 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.795240 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.796313 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.796718 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.796820 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.796833 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.797006 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.797051 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:46:42.797410 17314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.793895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:42739 (local address 127.20.105.67:42223)
0808 00:46:42.794010 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.794029 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:42.794091 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:42.794341 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:42.794342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:42.794344 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:42.794468 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.794471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.795268 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.795269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.796352 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:42.796355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.797093 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.797094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:42.797163 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.797166 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:46:42.797243 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 00:46:42.797303 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":48}
I20260808 00:46:42.797560 17339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.794017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:42223 (local address 127.20.105.66:42739)
0808 00:46:42.794093 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.794095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.794110 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:46:42.794235 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.794237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.794248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.794333 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:42.794561 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.795153 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.796473 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.796956 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.797036 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.797047 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.797419 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.797479 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:46:42.797561 16874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b96a23cfc00641598ecd0913ea850a3a" candidate_uuid: "c99c146127fa45a59fe11d6f710a69dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" is_pre_election: true
I20260808 00:46:42.797770 16874 raft_consensus.cc:2454] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c99c146127fa45a59fe11d6f710a69dd in term 0.
I20260808 00:46:42.797919 17064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b96a23cfc00641598ecd0913ea850a3a" candidate_uuid: "c99c146127fa45a59fe11d6f710a69dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" is_pre_election: true
I20260808 00:46:42.798058 17162 leader_election.cc:302] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016a6ca30eef43c4885c20c45d5ff3c3, c99c146127fa45a59fe11d6f710a69dd; no voters:
I20260808 00:46:42.798122 17064 raft_consensus.cc:2454] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c99c146127fa45a59fe11d6f710a69dd in term 0.
I20260808 00:46:42.798239 17359 raft_consensus.cc:2781] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:46:42.798295 17359 raft_consensus.cc:482] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:46:42.798319 17359 raft_consensus.cc:3037] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:42.799250 17359 raft_consensus.cc:504] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.799383 17359 leader_election.cc:288] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [CANDIDATE]: Term 1 election: Requested vote from peers 0ab9fac3247b4b0e9ec932a1a8cd7d04 (127.20.105.66:42739), 016a6ca30eef43c4885c20c45d5ff3c3 (127.20.105.65:38517)
I20260808 00:46:42.799630 17064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b96a23cfc00641598ecd0913ea850a3a" candidate_uuid: "c99c146127fa45a59fe11d6f710a69dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04"
I20260808 00:46:42.799652 16874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b96a23cfc00641598ecd0913ea850a3a" candidate_uuid: "c99c146127fa45a59fe11d6f710a69dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016a6ca30eef43c4885c20c45d5ff3c3"
I20260808 00:46:42.799731 17064 raft_consensus.cc:3037] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:42.799732 16874 raft_consensus.cc:3037] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:42.800858 17064 raft_consensus.cc:2454] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c99c146127fa45a59fe11d6f710a69dd in term 1.
I20260808 00:46:42.800858 16874 raft_consensus.cc:2454] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c99c146127fa45a59fe11d6f710a69dd in term 1.
I20260808 00:46:42.801101 17165 leader_election.cc:302] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ab9fac3247b4b0e9ec932a1a8cd7d04, c99c146127fa45a59fe11d6f710a69dd; no voters:
I20260808 00:46:42.801213 17359 raft_consensus.cc:2781] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:42.801410 17359 raft_consensus.cc:686] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 1 LEADER]: Becoming Leader. State: Replica: c99c146127fa45a59fe11d6f710a69dd, State: Running, Role: LEADER
I20260808 00:46:42.801555 17359 consensus_queue.cc:237] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } }
I20260808 00:46:42.802760 16669 catalog_manager.cc:5495] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd reported cstate change: term changed from 0 to 1, leader changed from <none> to c99c146127fa45a59fe11d6f710a69dd (127.20.105.67). New cstate: current_term: 1 leader_uuid: "c99c146127fa45a59fe11d6f710a69dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c99c146127fa45a59fe11d6f710a69dd" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 41423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 } health_report { overall_health: UNKNOWN } } }
I20260808 00:46:42.810766 17342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.808484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.20.105.67:41423)
0808 00:46:42.808547 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:42.808549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:42.808559 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:46:42.808617 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:42.808618 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:42.808635 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:42.808695 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:46:42.808812 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.809290 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.810061 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:42.810243 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:42.810297 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:42.810430 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:46:42.810546 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:46:42.810608 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:42.810643 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 00:46:42.810659 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:46:42.813082 16874 raft_consensus.cc:1261] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 1 FOLLOWER]: Refusing update from remote peer c99c146127fa45a59fe11d6f710a69dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:42.813082 17064 raft_consensus.cc:1261] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 1 FOLLOWER]: Refusing update from remote peer c99c146127fa45a59fe11d6f710a69dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:42.813493 17359 consensus_queue.cc:1035] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ab9fac3247b4b0e9ec932a1a8cd7d04" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 42739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:42.813661 17361 consensus_queue.cc:1035] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "016a6ca30eef43c4885c20c45d5ff3c3" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 38517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:42.816884 17364 mvcc.cc:205] Tried to move back new op lower bound from 7316070411519066112 to 7316070411476324352. Current Snapshot: MvccSnapshot[applied={T|T < 7316070411519066112}]
I20260808 00:46:42.817937 17367 mvcc.cc:205] Tried to move back new op lower bound from 7316070411519066112 to 7316070411476324352. Current Snapshot: MvccSnapshot[applied={T|T < 7316070411519066112}]
I20260808 00:46:42.818603 17365 mvcc.cc:205] Tried to move back new op lower bound from 7316070411519066112 to 7316070411476324352. Current Snapshot: MvccSnapshot[applied={T|T < 7316070411519066112 or (T in {7316070411519066112})}]
I20260808 00:46:42.820060 16671 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:42.820192 16671 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:42.821193 16671 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 25a64cac54f94abca59f0715b92b8c4c: Sending DeleteTablet for 3 replicas of tablet b96a23cfc00641598ecd0913ea850a3a
I20260808 00:46:42.821619 16847 tablet_service.cc:1514] Processing DeleteTablet for tablet b96a23cfc00641598ecd0913ea850a3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:42 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:33302
I20260808 00:46:42.821610 17035 tablet_service.cc:1514] Processing DeleteTablet for tablet b96a23cfc00641598ecd0913ea850a3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:42 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:59484
I20260808 00:46:42.821877 17377 tablet_replica.cc:330] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3: stopping tablet replica
I20260808 00:46:42.822005 17377 raft_consensus.cc:2229] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:42.822052 17221 tablet_service.cc:1514] Processing DeleteTablet for tablet b96a23cfc00641598ecd0913ea850a3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:42 UTC) from {username='kudu', principal='kudu/127.20.105.126@KRBTEST.COM'} at 127.0.0.1:52638
I20260808 00:46:42.822156 17377 raft_consensus.cc:2258] T b96a23cfc00641598ecd0913ea850a3a P 016a6ca30eef43c4885c20c45d5ff3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:42.822363 17376 tablet_replica.cc:330] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: stopping tablet replica
I20260808 00:46:42.822474 17376 raft_consensus.cc:2229] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:42.822516 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 16759
I20260808 00:46:42.822576 17376 raft_consensus.cc:2258] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:42.822739 17378 tablet_replica.cc:330] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: stopping tablet replica
I20260808 00:46:42.822842 17378 raft_consensus.cc:2229] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:46:42.822953 17378 raft_consensus.cc:2258] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:42.823486 17378 ts_tablet_manager.cc:1905] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:42.825611 17378 ts_tablet_manager.cc:1918] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:42.825757 17378 log.cc:1196] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/b96a23cfc00641598ecd0913ea850a3a
I20260808 00:46:42.826169 17378 ts_tablet_manager.cc:1939] T b96a23cfc00641598ecd0913ea850a3a P c99c146127fa45a59fe11d6f710a69dd: Deleting consensus metadata
I20260808 00:46:42.834904 16655 catalog_manager.cc:4841] TS c99c146127fa45a59fe11d6f710a69dd (127.20.105.67:41423): tablet b96a23cfc00641598ecd0913ea850a3a (table test-table [id=8d3b929df1c84f989a02d415f975f178]) successfully deleted
I20260808 00:46:42.834935 17376 ts_tablet_manager.cc:1905] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:42.837196 17376 ts_tablet_manager.cc:1918] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:42.837343 17376 log.cc:1196] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/b96a23cfc00641598ecd0913ea850a3a
I20260808 00:46:42.837659 17376 ts_tablet_manager.cc:1939] T b96a23cfc00641598ecd0913ea850a3a P 0ab9fac3247b4b0e9ec932a1a8cd7d04: Deleting consensus metadata
I20260808 00:46:42.838274 16655 catalog_manager.cc:4841] TS 0ab9fac3247b4b0e9ec932a1a8cd7d04 (127.20.105.66:42739): tablet b96a23cfc00641598ecd0913ea850a3a (table test-table [id=8d3b929df1c84f989a02d415f975f178]) successfully deleted
W20260808 00:46:42.843899 16653 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:38517 (error 108)
I20260808 00:46:42.844245 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 16947
I20260808 00:46:42.847770 17333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:42.847522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:38517 (local address 127.0.0.1:33304)
0808 00:46:42.847631 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:42.847702 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:38517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260808 00:46:42.847981 16653 catalog_manager.cc:4584] TS 016a6ca30eef43c4885c20c45d5ff3c3 (127.20.105.65:38517): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b96a23cfc00641598ecd0913ea850a3a: Network error: Client connection negotiation failed: client connection to 127.20.105.65:38517: connect: Connection refused (error 111)
I20260808 00:46:42.860647 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 17134
I20260808 00:46:42.876366 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 16629
2026-08-08T00:46:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2491 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T00:46:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:42Z Disabled control of system clock
I20260808 00:46:42.918210 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34357
--webserver_interface=127.20.105.126
--webserver_port=0
--builtin_ntp_servers=127.20.105.84:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:34357
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:43.019408 17383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:43.019793 17383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:43.019820 17383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:43.021397 17383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:43.021474 17383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:43.021489 17383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:43.021503 17383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:43.021517 17383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:43.023212 17383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:34357
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:34357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:43.023540 17383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:43.023926 17383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:43.030186 17389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:43.030282 17391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:43.030289 17394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:43.031025 17383 server_base.cc:1034] running on GCE node
I20260808 00:46:43.031396 17383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:43.031786 17383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:43.032972 17383 hybrid_clock.cc:648] HybridClock initialized: now 1786150003032929 us; error 58 us; skew 500 ppm
I20260808 00:46:43.034715 17383 webserver.cc:460] Webserver started at http://127.20.105.126:40097/ using document root <none> and password file <none>
I20260808 00:46:43.035004 17383 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:43.035053 17383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:43.035168 17383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:43.036617 17383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "afde2b5d04784c9db9d5f91fa9ea737e"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "68d8b9df52b2c0f77b7836a023841207"
server_key_iv: "6a8f754f5d3bd9cee73efa5529d5810f"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.036998 17383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "afde2b5d04784c9db9d5f91fa9ea737e"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "68d8b9df52b2c0f77b7836a023841207"
server_key_iv: "6a8f754f5d3bd9cee73efa5529d5810f"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.038552 17383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:46:43.039530 17404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.039806 17383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:43.039937 17383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "afde2b5d04784c9db9d5f91fa9ea737e"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "68d8b9df52b2c0f77b7836a023841207"
server_key_iv: "6a8f754f5d3bd9cee73efa5529d5810f"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.040084 17383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:43.050154 17383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:43.050625 17383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:43.050833 17383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:43.056559 17383 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:34357
I20260808 00:46:43.056627 17483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:34357 every 8 connection(s)
I20260808 00:46:43.057087 17383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:43.058337 17484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:43.061342 17484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e: Bootstrap starting.
I20260808 00:46:43.062135 17484 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:43.062536 17484 log.cc:824] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:43.063565 17484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e: No bootstrap required, opened a new log
I20260808 00:46:43.065033 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 17383
I20260808 00:46:43.065178 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:43.065414 20901 external_mini_cluster.cc:1428] Setting key 42f293f57898eadd51521c8a09ae382d
I20260808 00:46:43.066324 17484 raft_consensus.cc:348] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afde2b5d04784c9db9d5f91fa9ea737e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34357 } }
I20260808 00:46:43.066617 17484 raft_consensus.cc:374] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:43.066671 17484 raft_consensus.cc:729] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afde2b5d04784c9db9d5f91fa9ea737e, State: Initialized, Role: FOLLOWER
I20260808 00:46:43.066861 17484 consensus_queue.cc:260] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afde2b5d04784c9db9d5f91fa9ea737e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34357 } }
I20260808 00:46:43.066967 17484 raft_consensus.cc:388] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:43.067005 17484 raft_consensus.cc:482] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:43.067054 17484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:43.068035 17484 raft_consensus.cc:504] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afde2b5d04784c9db9d5f91fa9ea737e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34357 } }
I20260808 00:46:43.068176 17484 leader_election.cc:302] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afde2b5d04784c9db9d5f91fa9ea737e; no voters:
I20260808 00:46:43.068375 17484 leader_election.cc:288] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:43.068650 17491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:43.068746 17484 sys_catalog.cc:564] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:43.069643 17491 raft_consensus.cc:686] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [term 1 LEADER]: Becoming Leader. State: Replica: afde2b5d04784c9db9d5f91fa9ea737e, State: Running, Role: LEADER
I20260808 00:46:43.069769 17491 consensus_queue.cc:237] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afde2b5d04784c9db9d5f91fa9ea737e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34357 } }
I20260808 00:46:43.070485 17489 sys_catalog.cc:455] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afde2b5d04784c9db9d5f91fa9ea737e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afde2b5d04784c9db9d5f91fa9ea737e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34357 } } }
I20260808 00:46:43.070595 17489 sys_catalog.cc:458] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:43.070793 17494 sys_catalog.cc:455] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [sys.catalog]: SysCatalogTable state changed. Reason: New leader afde2b5d04784c9db9d5f91fa9ea737e. Latest consensus state: current_term: 1 leader_uuid: "afde2b5d04784c9db9d5f91fa9ea737e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afde2b5d04784c9db9d5f91fa9ea737e" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 34357 } } }
I20260808 00:46:43.071399 17494 sys_catalog.cc:458] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:43.070858 17508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:43.072010 17508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:43.074180 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.066216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.20.105.126:34357)
0808 00:46:43.066783 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.066794 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.066846 (+ 52us) server_negotiation.cc:407] Connection header received
0808 00:46:43.067003 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.067013 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.067102 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.067294 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:43.067656 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.069839 (+ 2183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.071151 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.071851 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.072011 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.073000 (+ 989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:43.073004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:43.073004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:43.073211 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:43.073252 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:43.073265 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:43.073350 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.073777 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.073902 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":290,"thread_start_us":160,"threads_started":1}
I20260808 00:46:43.074177 17508 catalog_manager.cc:1347] Generated new cluster ID: ed3f27221d6948b0a46d719894fa3d94
I20260808 00:46:43.074272 17508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:43.098613 17508 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:43.099488 17508 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:43.104875 17508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e: Generated new TSK 0
I20260808 00:46:43.105175 17508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:43.108346 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.65:0
--local_ip_for_outbound_sockets=127.20.105.65
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34357
--builtin_ntp_servers=127.20.105.84:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:43.213093 17515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:43.213407 17515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:43.213438 17515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:43.215122 17515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:43.215197 17515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:43.215236 17515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.65
I20260808 00:46:43.217325 17515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.105.65
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.65
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:43.217724 17515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:43.218170 17515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:43.224450 17528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:43.224427 17524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:43.224427 17522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:43.225190 17515 server_base.cc:1034] running on GCE node
I20260808 00:46:43.225466 17515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:43.225816 17515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:43.226989 17515 hybrid_clock.cc:648] HybridClock initialized: now 1786150003226966 us; error 46 us; skew 500 ppm
I20260808 00:46:43.228739 17515 webserver.cc:460] Webserver started at http://127.20.105.65:43067/ using document root <none> and password file <none>
I20260808 00:46:43.229094 17515 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:43.229149 17515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:43.229305 17515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:43.230881 17515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/data/instance:
uuid: "61562229ef7b4e8facca932ead602ac0"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "b1a47c1194725b55cc7821539bc5cc90"
server_key_iv: "3938ec0f59c1b54d00bf99083f38c58c"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.231519 17515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance:
uuid: "61562229ef7b4e8facca932ead602ac0"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "b1a47c1194725b55cc7821539bc5cc90"
server_key_iv: "3938ec0f59c1b54d00bf99083f38c58c"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.233660 17515 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260808 00:46:43.234716 17535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.235131 17515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:43.235266 17515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/wal
uuid: "61562229ef7b4e8facca932ead602ac0"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "b1a47c1194725b55cc7821539bc5cc90"
server_key_iv: "3938ec0f59c1b54d00bf99083f38c58c"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.235366 17515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:43.279987 17515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:43.280544 17515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:43.280687 17515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:43.280941 17515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:43.281427 17515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:43.281491 17515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.281543 17515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:43.281571 17515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.289129 17515 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.65:35119
I20260808 00:46:43.289181 17706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.65:35119 every 8 connection(s)
I20260808 00:46:43.289626 17515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/data/info.pb
I20260808 00:46:43.296304 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.290708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.65:40665 (local address 127.20.105.126:34357)
0808 00:46:43.290810 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.290813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.291106 (+ 293us) server_negotiation.cc:407] Connection header received
0808 00:46:43.291775 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.291777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.291795 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.291861 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:43.292751 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.293329 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.294521 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.294742 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.294802 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.295920 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:43.295922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:43.295923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:43.295928 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:43.295948 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:43.295953 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:43.296002 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.296180 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.296219 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:46:43.296550 17709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.290674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34357 (local address 127.20.105.65:40665)
0808 00:46:43.290957 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.291001 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:43.291643 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:43.291982 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:43.291987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:43.291997 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:43.292654 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.292659 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.293450 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.293452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.294427 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.294430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.294833 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.294835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.295012 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.295741 (+ 729us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:43.295742 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:43.295779 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:43.295806 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:43.296081 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:43.296095 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:43.296098 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:46:43.296169 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:46:43.296275 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":116,"threads_started":1}
I20260808 00:46:43.297263 17707 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34357
I20260808 00:46:43.297391 17707 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:43.297652 17707 heartbeater.cc:511] Master 127.20.105.126:34357 requested a full tablet report, sending...
I20260808 00:46:43.297958 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 17515
I20260808 00:46:43.298101 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-0/wal/instance
I20260808 00:46:43.298342 20901 external_mini_cluster.cc:1428] Setting key 9b8e563bbe58717fe6520b79b1efe6ba
I20260808 00:46:43.298388 17428 ts_manager.cc:194] Registered new tserver with Master: 61562229ef7b4e8facca932ead602ac0 (127.20.105.65:35119)
I20260808 00:46:43.299520 17428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.65:40665
I20260808 00:46:43.299749 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.66:0
--local_ip_for_outbound_sockets=127.20.105.66
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34357
--builtin_ntp_servers=127.20.105.84:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:43.397329 17710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:43.397585 17710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:43.397611 17710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:43.398914 17710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:43.398975 17710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:43.399003 17710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.66
I20260808 00:46:43.400604 17710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.105.66
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.66
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:43.400928 17710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:43.401321 17710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:43.407464 17720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:43.407511 17724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:43.407459 17710 server_base.cc:1034] running on GCE node
W20260808 00:46:43.407500 17718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:43.407963 17710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:43.408327 17710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:43.409488 17710 hybrid_clock.cc:648] HybridClock initialized: now 1786150003409470 us; error 43 us; skew 500 ppm
I20260808 00:46:43.411070 17710 webserver.cc:460] Webserver started at http://127.20.105.66:38015/ using document root <none> and password file <none>
I20260808 00:46:43.411373 17710 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:43.411423 17710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:43.411584 17710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:43.413007 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/data/instance:
uuid: "c8661e40b1734046afa1ad4ae7cc25f7"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "073f94fdeba7b78a693d27a19a32dfa5"
server_key_iv: "bf3b55c71e19679baa10c4f574b86f56"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.413439 17710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance:
uuid: "c8661e40b1734046afa1ad4ae7cc25f7"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "073f94fdeba7b78a693d27a19a32dfa5"
server_key_iv: "bf3b55c71e19679baa10c4f574b86f56"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.415030 17710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:46:43.415902 17730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.416148 17710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:43.416265 17710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/wal
uuid: "c8661e40b1734046afa1ad4ae7cc25f7"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "073f94fdeba7b78a693d27a19a32dfa5"
server_key_iv: "bf3b55c71e19679baa10c4f574b86f56"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.416355 17710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:43.428762 17710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:43.429206 17710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:43.429380 17710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:43.429678 17710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:43.430091 17710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:43.430150 17710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.430197 17710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:43.430238 17710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.438190 17710 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.66:39919
I20260808 00:46:43.438313 17899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.66:39919 every 8 connection(s)
I20260808 00:46:43.438755 17710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/data/info.pb
I20260808 00:46:43.445508 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.439868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:38555 (local address 127.20.105.126:34357)
0808 00:46:43.440029 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.440031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.440276 (+ 245us) server_negotiation.cc:407] Connection header received
0808 00:46:43.440975 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.440976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.440992 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.441058 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:43.441908 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.442520 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.443741 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.443976 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.444104 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.445146 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:43.445147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:43.445148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:43.445155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:43.445183 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:43.445188 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:43.445252 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.445357 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.445400 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 00:46:43.445842 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.439817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34357 (local address 127.20.105.66:38555)
0808 00:46:43.440129 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.440173 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:43.440846 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:43.441154 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:43.441163 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:43.441173 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:43.441801 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.441805 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.442647 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.442649 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.443637 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.443640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.444117 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.444118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.444282 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.444975 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:43.444976 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:43.445015 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:43.445044 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:43.445254 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:43.445262 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:43.445266 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:46:43.445361 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 00:46:43.445533 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":118,"threads_started":1}
I20260808 00:46:43.446525 17900 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34357
I20260808 00:46:43.446641 17900 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:43.446856 17900 heartbeater.cc:511] Master 127.20.105.126:34357 requested a full tablet report, sending...
I20260808 00:46:43.447359 17428 ts_manager.cc:194] Registered new tserver with Master: c8661e40b1734046afa1ad4ae7cc25f7 (127.20.105.66:39919)
I20260808 00:46:43.447777 17428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.66:38555
I20260808 00:46:43.448630 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 17710
I20260808 00:46:43.448731 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/wal/instance
I20260808 00:46:43.448997 20901 external_mini_cluster.cc:1428] Setting key 2d15bed7c18d9da043170d8bb018f58f
I20260808 00:46:43.450057 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.105.67:0
--local_ip_for_outbound_sockets=127.20.105.67
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34357
--builtin_ntp_servers=127.20.105.84:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:46:43.549135 17904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:43.549441 17904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:43.549479 17904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:43.550750 17904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:43.550817 17904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:43.550860 17904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.105.67
I20260808 00:46:43.552423 17904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.105.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.105.67
--webserver_port=0
--tserver_master_addrs=127.20.105.126:34357
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.105.67
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:43.552733 17904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:43.553107 17904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:43.559058 17917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:43.559026 17916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:43.559034 17921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:43.559478 17904 server_base.cc:1034] running on GCE node
I20260808 00:46:43.559753 17904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:43.560000 17904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:43.561219 17904 hybrid_clock.cc:648] HybridClock initialized: now 1786150003561200 us; error 49 us; skew 500 ppm
I20260808 00:46:43.562827 17904 webserver.cc:460] Webserver started at http://127.20.105.67:46021/ using document root <none> and password file <none>
I20260808 00:46:43.563081 17904 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:43.563131 17904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:43.563271 17904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:43.564692 17904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/data/instance:
uuid: "ce6dca089e044316972a45d878eda472"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "73c25aff75adcd47eeb29014b83e101e"
server_key_iv: "a3ed3f074886694850b91add5a9a9f99"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.565061 17904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance:
uuid: "ce6dca089e044316972a45d878eda472"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "73c25aff75adcd47eeb29014b83e101e"
server_key_iv: "a3ed3f074886694850b91add5a9a9f99"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.566548 17904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:46:43.567390 17928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.567625 17904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:43.567727 17904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/wal
uuid: "ce6dca089e044316972a45d878eda472"
format_stamp: "Formatted at 2026-08-08 00:46:43 on dist-test-slave-kvfs"
server_key: "73c25aff75adcd47eeb29014b83e101e"
server_key_iv: "a3ed3f074886694850b91add5a9a9f99"
server_key_version: "encryptionkey@0"
I20260808 00:46:43.567795 17904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:43.577528 17904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:43.577862 17904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:43.577986 17904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:43.578217 17904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:46:43.578564 17904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:46:43.578598 17904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.578624 17904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:46:43.578637 17904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:43.585566 17904 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.67:45793
I20260808 00:46:43.585641 18117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.67:45793 every 8 connection(s)
I20260808 00:46:43.586058 17904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/data/info.pb
I20260808 00:46:43.587875 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 17904
I20260808 00:46:43.588027 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/wal/instance
I20260808 00:46:43.588304 20901 external_mini_cluster.cc:1428] Setting key 59e870d55f87e76dc498ba3e92143a34
I20260808 00:46:43.592878 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.587043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.67:34877 (local address 127.20.105.126:34357)
0808 00:46:43.587143 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.587147 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.587407 (+ 260us) server_negotiation.cc:407] Connection header received
0808 00:46:43.588091 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.588093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.588110 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.588168 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:43.589332 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.590096 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.591183 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.591386 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.591449 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.592527 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:43.592529 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:43.592529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:43.592534 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:43.592555 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:43.592559 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:43.592612 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.592758 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.592794 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 00:46:43.593127 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.586997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:34357 (local address 127.20.105.67:34877)
0808 00:46:43.587243 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.587286 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:43.587913 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:43.588319 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:43.588323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:43.588334 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:46:43.589222 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.589230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.590212 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.590213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.591093 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.591096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.591472 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.591473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.591598 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.592339 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:46:43.592340 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:46:43.592389 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:46:43.592417 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:46:43.592660 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:46:43.592667 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:46:43.592670 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:46:43.592753 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:46:43.592860 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":128,"threads_started":1}
I20260808 00:46:43.593758 18119 heartbeater.cc:347] Connected to a master server at 127.20.105.126:34357
I20260808 00:46:43.593866 18119 heartbeater.cc:464] Registering TS with master...
I20260808 00:46:43.594061 18119 heartbeater.cc:511] Master 127.20.105.126:34357 requested a full tablet report, sending...
I20260808 00:46:43.594544 17428 ts_manager.cc:194] Registered new tserver with Master: ce6dca089e044316972a45d878eda472 (127.20.105.67:45793)
I20260808 00:46:43.594918 17428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.105.67:34877
I20260808 00:46:43.601082 20901 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:46:43.606570 17488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.602947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.20.105.126:34357)
0808 00:46:43.603022 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.603024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.603069 (+ 45us) server_negotiation.cc:407] Connection header received
0808 00:46:43.603154 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.603154 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.603183 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.603263 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:43.603471 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.604044 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.604967 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.605261 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.605331 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.606225 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:43.606227 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:43.606227 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:43.606232 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:43.606251 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:43.606255 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:43.606307 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.606441 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.606484 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:46:43.608860 17428 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47142:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:46:43.611021 17428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:46:43.619228 18140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.614284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:35119 (local address 127.0.0.1:50758)
0808 00:46:43.614459 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.614533 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:43.614623 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:43.615123 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:43.615127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:43.615130 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:43.615232 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.615237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.616529 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.616530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.617921 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.617925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.618809 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.618811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.618916 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.618933 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 00:46:43.619026 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 00:46:43.619098 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260808 00:46:43.619534 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.614698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.20.105.66:39919)
0808 00:46:43.614900 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.614906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.615301 (+ 395us) server_negotiation.cc:407] Connection header received
0808 00:46:43.615373 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.615376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.615395 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.615484 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:43.615616 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.616794 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.618056 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.618848 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.619247 (+ 399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.619356 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.619386 (+ 30us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.619435 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":117,"threads_started":1}
I20260808 00:46:43.619565 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.614439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41178 (local address 127.20.105.67:45793)
0808 00:46:43.615001 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.615005 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.615165 (+ 160us) server_negotiation.cc:407] Connection header received
0808 00:46:43.615478 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.615482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.615501 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.615590 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:43.615825 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.616975 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.618202 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.618914 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.618990 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.619089 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.619421 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.619471 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":120,"threads_started":1}
I20260808 00:46:43.619212 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.614691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.20.105.65:35119)
0808 00:46:43.614900 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.614906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.614922 (+ 16us) server_negotiation.cc:407] Connection header received
0808 00:46:43.614977 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.614979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.614998 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.615102 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:43.615344 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.616428 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.618046 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.618696 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.618808 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.618936 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.619044 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.619096 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":121,"threads_started":1}
I20260808 00:46:43.619340 18138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.614283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:45793 (local address 127.0.0.1:41178)
0808 00:46:43.615090 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.615102 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:43.615354 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:43.615678 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:43.615679 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:43.615682 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:43.615732 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.615734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.617077 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.617078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.618107 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.618110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.619123 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.619124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.619176 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.619179 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:46:43.619226 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 00:46:43.619265 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":790,"mutex_wait_us":5,"thread_start_us":98,"threads_started":1}
I20260808 00:46:43.620134 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.614548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.66:39919 (local address 127.0.0.1:59520)
0808 00:46:43.615218 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.615227 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:43.615293 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:43.615481 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:43.615481 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:43.615484 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:43.615555 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.615558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.616888 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.616889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.617983 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.617986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.618999 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.619001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.619064 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.619070 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:46:43.619115 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:46:43.619164 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"thread_start_us":75,"threads_started":1}
I20260808 00:46:43.620942 17800 tablet_service.cc:1467] Processing CreateTablet for tablet 1023a4f42a3f4d65b73dddfbfca96ffb (DEFAULT_TABLE table=test-table [id=b279398229564296acd60bee7d9e45c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:43.620935 18006 tablet_service.cc:1467] Processing CreateTablet for tablet 1023a4f42a3f4d65b73dddfbfca96ffb (DEFAULT_TABLE table=test-table [id=b279398229564296acd60bee7d9e45c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:43.620930 17609 tablet_service.cc:1467] Processing CreateTablet for tablet 1023a4f42a3f4d65b73dddfbfca96ffb (DEFAULT_TABLE table=test-table [id=b279398229564296acd60bee7d9e45c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:46:43.621402 17800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1023a4f42a3f4d65b73dddfbfca96ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:43.621399 17609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1023a4f42a3f4d65b73dddfbfca96ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:43.621399 18006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1023a4f42a3f4d65b73dddfbfca96ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:43.625057 18146 tablet_bootstrap.cc:492] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Bootstrap starting.
I20260808 00:46:43.625408 18149 tablet_bootstrap.cc:492] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: Bootstrap starting.
I20260808 00:46:43.625540 18148 tablet_bootstrap.cc:492] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0: Bootstrap starting.
I20260808 00:46:43.626101 18146 tablet_bootstrap.cc:654] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:43.626209 18148 tablet_bootstrap.cc:654] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:43.626210 18149 tablet_bootstrap.cc:654] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:43.626540 18146 log.cc:824] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:43.626595 18149 log.cc:824] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:43.626595 18148 log.cc:824] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:43.627372 18149 tablet_bootstrap.cc:492] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: No bootstrap required, opened a new log
I20260808 00:46:43.627381 18148 tablet_bootstrap.cc:492] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0: No bootstrap required, opened a new log
I20260808 00:46:43.627460 18148 ts_tablet_manager.cc:1397] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:46:43.627440 18149 ts_tablet_manager.cc:1397] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:46:43.628489 18146 tablet_bootstrap.cc:492] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: No bootstrap required, opened a new log
I20260808 00:46:43.628566 18146 ts_tablet_manager.cc:1397] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 00:46:43.629698 18149 raft_consensus.cc:348] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.629884 18149 raft_consensus.cc:374] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:43.629930 18149 raft_consensus.cc:729] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce6dca089e044316972a45d878eda472, State: Initialized, Role: FOLLOWER
I20260808 00:46:43.630077 18149 consensus_queue.cc:260] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.630417 18149 ts_tablet_manager.cc:1428] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:46:43.630376 18148 raft_consensus.cc:348] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.630563 18119 heartbeater.cc:503] Master 127.20.105.126:34357 was elected leader, sending a full tablet report...
I20260808 00:46:43.630568 18148 raft_consensus.cc:374] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:43.630609 18148 raft_consensus.cc:729] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61562229ef7b4e8facca932ead602ac0, State: Initialized, Role: FOLLOWER
I20260808 00:46:43.630722 18148 consensus_queue.cc:260] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.631007 18148 ts_tablet_manager.cc:1428] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:46:43.631168 17707 heartbeater.cc:503] Master 127.20.105.126:34357 was elected leader, sending a full tablet report...
I20260808 00:46:43.631738 18146 raft_consensus.cc:348] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.631940 18146 raft_consensus.cc:374] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:43.631985 18146 raft_consensus.cc:729] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8661e40b1734046afa1ad4ae7cc25f7, State: Initialized, Role: FOLLOWER
I20260808 00:46:43.632211 18146 consensus_queue.cc:260] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.632519 18146 ts_tablet_manager.cc:1428] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:46:43.632586 17900 heartbeater.cc:503] Master 127.20.105.126:34357 was elected leader, sending a full tablet report...
W20260808 00:46:43.689613 17901 tablet.cc:2369] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:46:43.699749 18157 raft_consensus.cc:482] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:46:43.699942 18157 raft_consensus.cc:504] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.700559 18157 leader_election.cc:288] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce6dca089e044316972a45d878eda472 (127.20.105.67:45793), 61562229ef7b4e8facca932ead602ac0 (127.20.105.65:35119)
I20260808 00:46:43.705057 18141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.700992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:60649 (local address 127.20.105.67:45793)
0808 00:46:43.701051 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.701054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.701203 (+ 149us) server_negotiation.cc:407] Connection header received
0808 00:46:43.701265 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.701266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.701277 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.701343 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:43.701524 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.702309 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.704053 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.704643 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.704730 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.704741 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.704924 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.704976 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:46:43.705067 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.700988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.105.66:53847 (local address 127.20.105.65:35119)
0808 00:46:43.701054 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.701056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.701066 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:46:43.701117 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.701118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.701137 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.701234 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:46:43.701494 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.702309 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.704053 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.704645 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.704730 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.704741 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.704953 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.705000 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:46:43.705060 17902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.700844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:35119 (local address 127.20.105.66:53847)
0808 00:46:43.700931 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.700947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:43.701005 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:43.701252 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:43.701254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:43.701256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:43.701371 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.701373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.702407 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.702408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.703902 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.703907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.704744 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.704745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.704836 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.704841 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:43.704905 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 00:46:43.704969 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":68}
I20260808 00:46:43.705066 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.700844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.67:45793 (local address 127.20.105.66:60649)
0808 00:46:43.701117 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.701128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:46:43.701183 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:46:43.701356 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:46:43.701357 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:46:43.701358 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:46:43.701443 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.701445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.702407 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.702408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.703902 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:46:43.703907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.704744 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.704745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:46:43.704836 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.704841 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:46:43.704905 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 00:46:43.704954 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":109,"thread_start_us":199,"threads_started":1}
I20260808 00:46:43.705566 18043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1023a4f42a3f4d65b73dddfbfca96ffb" candidate_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce6dca089e044316972a45d878eda472" is_pre_election: true
I20260808 00:46:43.705565 17634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1023a4f42a3f4d65b73dddfbfca96ffb" candidate_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61562229ef7b4e8facca932ead602ac0" is_pre_election: true
I20260808 00:46:43.705771 18043 raft_consensus.cc:2454] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8661e40b1734046afa1ad4ae7cc25f7 in term 0.
I20260808 00:46:43.705770 17634 raft_consensus.cc:2454] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c8661e40b1734046afa1ad4ae7cc25f7 in term 0.
I20260808 00:46:43.706071 17731 leader_election.cc:302] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 61562229ef7b4e8facca932ead602ac0, c8661e40b1734046afa1ad4ae7cc25f7, ce6dca089e044316972a45d878eda472; no voters:
I20260808 00:46:43.706225 18157 raft_consensus.cc:2781] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:46:43.706290 18157 raft_consensus.cc:482] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:46:43.706317 18157 raft_consensus.cc:3037] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:43.707223 18157 raft_consensus.cc:504] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.707393 18157 leader_election.cc:288] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [CANDIDATE]: Term 1 election: Requested vote from peers ce6dca089e044316972a45d878eda472 (127.20.105.67:45793), 61562229ef7b4e8facca932ead602ac0 (127.20.105.65:35119)
I20260808 00:46:43.707556 18043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1023a4f42a3f4d65b73dddfbfca96ffb" candidate_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce6dca089e044316972a45d878eda472"
I20260808 00:46:43.707607 17634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1023a4f42a3f4d65b73dddfbfca96ffb" candidate_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61562229ef7b4e8facca932ead602ac0"
I20260808 00:46:43.707677 18043 raft_consensus.cc:3037] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:43.707731 17634 raft_consensus.cc:3037] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:43.708751 18043 raft_consensus.cc:2454] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8661e40b1734046afa1ad4ae7cc25f7 in term 1.
I20260808 00:46:43.708915 17634 raft_consensus.cc:2454] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c8661e40b1734046afa1ad4ae7cc25f7 in term 1.
I20260808 00:46:43.708930 17731 leader_election.cc:302] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8661e40b1734046afa1ad4ae7cc25f7, ce6dca089e044316972a45d878eda472; no voters:
I20260808 00:46:43.709019 18157 raft_consensus.cc:2781] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:43.709220 18157 raft_consensus.cc:686] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 1 LEADER]: Becoming Leader. State: Replica: c8661e40b1734046afa1ad4ae7cc25f7, State: Running, Role: LEADER
I20260808 00:46:43.709350 18157 consensus_queue.cc:237] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } }
I20260808 00:46:43.710451 17428 catalog_manager.cc:5495] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to c8661e40b1734046afa1ad4ae7cc25f7 (127.20.105.66). New cstate: current_term: 1 leader_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8661e40b1734046afa1ad4ae7cc25f7" member_type: VOTER last_known_addr { host: "127.20.105.66" port: 39919 } health_report { overall_health: HEALTHY } } }
I20260808 00:46:43.737627 18145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.734728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59522 (local address 127.20.105.66:39919)
0808 00:46:43.734817 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:43.734819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:43.734829 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:46:43.734890 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:43.734892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:43.734917 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:43.735043 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:46:43.735216 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.735768 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.736685 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:43.736906 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:43.736994 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:43.737179 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:46:43.737322 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:46:43.737432 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:43.737451 (+ 19us) server_negotiation.cc:299] Negotiation successful
0808 00:46:43.737478 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 00:46:43.740317 17634 raft_consensus.cc:1261] T 1023a4f42a3f4d65b73dddfbfca96ffb P 61562229ef7b4e8facca932ead602ac0 [term 1 FOLLOWER]: Refusing update from remote peer c8661e40b1734046afa1ad4ae7cc25f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:43.740317 18043 raft_consensus.cc:1261] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 1 FOLLOWER]: Refusing update from remote peer c8661e40b1734046afa1ad4ae7cc25f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:46:43.740823 18159 consensus_queue.cc:1035] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61562229ef7b4e8facca932ead602ac0" member_type: VOTER last_known_addr { host: "127.20.105.65" port: 35119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:43.741024 18157 consensus_queue.cc:1035] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce6dca089e044316972a45d878eda472" member_type: VOTER last_known_addr { host: "127.20.105.67" port: 45793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:46:43.744513 18163 mvcc.cc:205] Tried to move back new op lower bound from 7316070415316299776 to 7316070415194308608. Current Snapshot: MvccSnapshot[applied={T|T < 7316070415316299776}]
I20260808 00:46:43.747735 17428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:43.747860 17428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47142:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:46:43.748852 17428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afde2b5d04784c9db9d5f91fa9ea737e: Sending DeleteTablet for 3 replicas of tablet 1023a4f42a3f4d65b73dddfbfca96ffb
I20260808 00:46:43.749285 17800 tablet_service.cc:1514] Processing DeleteTablet for tablet 1023a4f42a3f4d65b73dddfbfca96ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:43 UTC) from {username='slave'} at 127.0.0.1:59520
I20260808 00:46:43.749316 18006 tablet_service.cc:1514] Processing DeleteTablet for tablet 1023a4f42a3f4d65b73dddfbfca96ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:43 UTC) from {username='slave'} at 127.0.0.1:41178
I20260808 00:46:43.749660 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 17515
I20260808 00:46:43.749768 18177 tablet_replica.cc:330] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: stopping tablet replica
I20260808 00:46:43.749765 17609 tablet_service.cc:1514] Processing DeleteTablet for tablet 1023a4f42a3f4d65b73dddfbfca96ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:46:43 UTC) from {username='slave'} at 127.0.0.1:50758
I20260808 00:46:43.749891 18177 raft_consensus.cc:2229] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:46:43.750029 18177 raft_consensus.cc:2258] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:43.750196 18176 tablet_replica.cc:330] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: stopping tablet replica
I20260808 00:46:43.750288 18176 raft_consensus.cc:2229] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:46:43.750442 18176 raft_consensus.cc:2258] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:46:43.758698 18177 ts_tablet_manager.cc:1905] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:43.761540 18177 ts_tablet_manager.cc:1918] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:43.761678 18177 log.cc:1196] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-2/wal/wals/1023a4f42a3f4d65b73dddfbfca96ffb
I20260808 00:46:43.762053 18177 ts_tablet_manager.cc:1939] T 1023a4f42a3f4d65b73dddfbfca96ffb P ce6dca089e044316972a45d878eda472: Deleting consensus metadata
I20260808 00:46:43.762650 18176 ts_tablet_manager.cc:1905] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:46:43.764405 18176 ts_tablet_manager.cc:1918] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:46:43.764528 18176 log.cc:1196] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Deleting WAL directory at /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149729697709-20901-0/minicluster-data/ts-1/wal/wals/1023a4f42a3f4d65b73dddfbfca96ffb
I20260808 00:46:43.764810 18176 ts_tablet_manager.cc:1939] T 1023a4f42a3f4d65b73dddfbfca96ffb P c8661e40b1734046afa1ad4ae7cc25f7: Deleting consensus metadata
I20260808 00:46:43.765475 17405 catalog_manager.cc:4841] TS c8661e40b1734046afa1ad4ae7cc25f7 (127.20.105.66:39919): tablet 1023a4f42a3f4d65b73dddfbfca96ffb (table test-table [id=b279398229564296acd60bee7d9e45c4]) successfully deleted
I20260808 00:46:43.765760 17405 catalog_manager.cc:4841] TS ce6dca089e044316972a45d878eda472 (127.20.105.67:45793): tablet 1023a4f42a3f4d65b73dddfbfca96ffb (table test-table [id=b279398229564296acd60bee7d9e45c4]) successfully deleted
W20260808 00:46:43.771821 17410 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.105.65:35119 (error 108)
I20260808 00:46:43.772222 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 17710
I20260808 00:46:43.777724 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:43.775790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.65:35119 (local address 127.0.0.1:50770)
0808 00:46:43.777575 (+ 1785us) negotiation.cc:107] Waiting for socket to connect
0808 00:46:43.777664 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.65:35119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1716}
W20260808 00:46:43.779629 17410 catalog_manager.cc:4584] TS 61562229ef7b4e8facca932ead602ac0 (127.20.105.65:35119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1023a4f42a3f4d65b73dddfbfca96ffb: Network error: Client connection negotiation failed: client connection to 127.20.105.65:35119: connect: Connection refused (error 111)
I20260808 00:46:43.784700 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 17904
I20260808 00:46:43.798190 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 17383
2026-08-08T00:46:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (922 ms)
[----------] 6 tests from EncryptionPolicyTest (10576 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:43 dist-test-slave-kvfs krb5kdc[18183](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:43 dist-test-slave-kvfs krb5kdc[18183](info): set up 2 sockets
Aug 08 00:46:43 dist-test-slave-kvfs krb5kdc[18183](info): commencing operation
krb5kdc: starting...
W20260808 00:46:45.089910 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:45 dist-test-slave-kvfs krb5kdc[18183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:45.249559 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32851
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32851
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:45.352999 18205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:45.353266 18205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:45.353292 18205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:45.354628 18205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:45.354677 18205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:45.354692 18205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:45.354705 18205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:45.354717 18205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:45.356456 18205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32851
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:45.356778 18205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:45.357069 18205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:45.362879 18219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:45.362861 18221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:45.362861 18218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:45.363350 18205 server_base.cc:1034] running on GCE node
I20260808 00:46:45.363683 18205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:45.364102 18205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:45.365269 18205 hybrid_clock.cc:648] HybridClock initialized: now 1786150005365247 us; error 46 us; skew 500 ppm
Aug 08 00:46:45 dist-test-slave-kvfs krb5kdc[18183](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150005, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:45.368263 18205 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:45.368846 18205 webserver.cc:460] Webserver started at http://127.0.0.1:40935/ using document root <none> and password file <none>
I20260808 00:46:45.369175 18205 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:45.369252 18205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:45.369411 18205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:45.370709 18205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "3b4484a77532446e94813dd2528450bd"
format_stamp: "Formatted at 2026-08-08 00:46:45 on dist-test-slave-kvfs"
server_key: "5d72dad98632a45a3cb128cf4365e456"
server_key_iv: "9c47e17968be2db54f9909a60b349a7b"
server_key_version: "encryptionkey@0"
I20260808 00:46:45.371160 18205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "3b4484a77532446e94813dd2528450bd"
format_stamp: "Formatted at 2026-08-08 00:46:45 on dist-test-slave-kvfs"
server_key: "5d72dad98632a45a3cb128cf4365e456"
server_key_iv: "9c47e17968be2db54f9909a60b349a7b"
server_key_version: "encryptionkey@0"
I20260808 00:46:45.372876 18205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:46:45.373930 18233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:45.374292 18205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:45.374409 18205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "3b4484a77532446e94813dd2528450bd"
format_stamp: "Formatted at 2026-08-08 00:46:45 on dist-test-slave-kvfs"
server_key: "5d72dad98632a45a3cb128cf4365e456"
server_key_iv: "9c47e17968be2db54f9909a60b349a7b"
server_key_version: "encryptionkey@0"
I20260808 00:46:45.374521 18205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:45.384783 18205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:45.387018 18205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:45.387202 18205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:45.392611 18205 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32851
I20260808 00:46:45.392652 18310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32851 every 8 connection(s)
I20260808 00:46:45.393077 18205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:45.394124 18313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:45.396958 18313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd: Bootstrap starting.
I20260808 00:46:45.397627 18313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:45.397903 18313 log.cc:824] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:45.398705 18313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd: No bootstrap required, opened a new log
I20260808 00:46:45.399680 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 18205
I20260808 00:46:45.399837 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:45.400137 20901 external_mini_cluster.cc:1428] Setting key 7758f0f3ac188e70169b02e5694fce7c
I20260808 00:46:45.401063 18313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4484a77532446e94813dd2528450bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32851 } }
I20260808 00:46:45.401262 18313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:45.401316 18313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b4484a77532446e94813dd2528450bd, State: Initialized, Role: FOLLOWER
I20260808 00:46:45.401489 18313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4484a77532446e94813dd2528450bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32851 } }
I20260808 00:46:45.401593 18313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:45.401633 18313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:45.401692 18313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:45.402987 18313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4484a77532446e94813dd2528450bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32851 } }
I20260808 00:46:45.403090 18313 leader_election.cc:302] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b4484a77532446e94813dd2528450bd; no voters:
I20260808 00:46:45.403299 18313 leader_election.cc:288] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:45.403373 18319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:45.403553 18319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [term 1 LEADER]: Becoming Leader. State: Replica: 3b4484a77532446e94813dd2528450bd, State: Running, Role: LEADER
I20260808 00:46:45.403642 18313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:45.403667 18319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4484a77532446e94813dd2528450bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32851 } }
I20260808 00:46:45.404098 18319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b4484a77532446e94813dd2528450bd. Latest consensus state: current_term: 1 leader_uuid: "3b4484a77532446e94813dd2528450bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4484a77532446e94813dd2528450bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32851 } } }
I20260808 00:46:45.404206 18319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:45.404117 18321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b4484a77532446e94813dd2528450bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b4484a77532446e94813dd2528450bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32851 } } }
I20260808 00:46:45.404419 18321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:45.405468 18332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:45.406494 18332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:45.408725 18332 catalog_manager.cc:1347] Generated new cluster ID: ef9e7e6acda6428fa0a6423b18af064a
I20260808 00:46:45.408793 18332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:46:45 dist-test-slave-kvfs krb5kdc[18183](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:46:45.413338 18318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:45.400874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35417 (local address 127.0.0.1:32851)
0808 00:46:45.401109 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:45.401119 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:45.401149 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:46:45.401269 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:45.401276 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:45.401347 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:45.401545 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:45.402556 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:45.404211 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:45.405640 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:45.406217 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:45.406362 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:45.409536 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:45.409541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:45.409541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:45.409556 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:45.411409 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:45.411863 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:45.411866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:45.411867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:45.411999 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:45.412336 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:45.412338 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:45.412338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:45.412629 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:45.412862 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:45.412959 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 00:46:45.413080 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":134,"threads_started":1}
I20260808 00:46:45.414858 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:46:45.424072 18332 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:45.424886 18332 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:45.429961 18332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b4484a77532446e94813dd2528450bd: Generated new TSK 0
I20260808 00:46:45.430259 18332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:45.432423 18318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:45.417731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54845 (local address 127.0.0.1:32851)
0808 00:46:45.417787 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:45.417789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:45.423483 (+ 5694us) server_negotiation.cc:407] Connection header received
0808 00:46:45.423556 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:45.423557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:45.423573 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:45.423662 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:45.424623 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:45.425492 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:45.426624 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:45.426989 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:45.427072 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:45.428879 (+ 1807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:45.428881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:45.428882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:45.428891 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:45.430914 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:45.431387 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:45.431388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:45.431388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:45.431464 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:45.431713 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:45.431714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:45.431714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:45.431870 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:45.431992 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:45.432226 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 00:46:45.432308 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":26}
I20260808 00:46:45.435329 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 18205
W20260808 00:46:45.463783 18179 connection.cc:535] client connection to 127.0.0.1:32851 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32851): Connection reset by peer (error 104)
2026-08-08T00:46:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:45 dist-test-slave-kvfs krb5kdc[18354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:45 dist-test-slave-kvfs krb5kdc[18354](info): set up 2 sockets
Aug 08 00:46:45 dist-test-slave-kvfs krb5kdc[18354](info): commencing operation
krb5kdc: starting...
W20260808 00:46:46.715881 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:46 dist-test-slave-kvfs krb5kdc[18354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150006, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:46.868713 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34059
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:46.972117 18370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:46.972434 18370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:46.972481 18370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:46.973815 18370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:46.973904 18370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:46.973930 18370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:46.973950 18370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:46.973985 18370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:46.975713 18370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34059
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:46.976143 18370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:46.976457 18370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:46.982461 18380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:46.982484 18379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:46.982659 18370 server_base.cc:1034] running on GCE node
W20260808 00:46:46.982514 18383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:46.983105 18370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:46.983472 18370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:46.984644 18370 hybrid_clock.cc:648] HybridClock initialized: now 1786150006984626 us; error 45 us; skew 500 ppm
Aug 08 00:46:46 dist-test-slave-kvfs krb5kdc[18354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150006, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:46.987691 18370 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:46.988297 18370 webserver.cc:460] Webserver started at http://127.0.0.1:46109/ using document root <none> and password file <none>
I20260808 00:46:46.988569 18370 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:46.988642 18370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:46.988813 18370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:46.990092 18370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "9828a9886f044a8c8ec7f4f2ecf29876"
format_stamp: "Formatted at 2026-08-08 00:46:46 on dist-test-slave-kvfs"
server_key: "f276913e1c3f6f27454f9df6e83b1d70"
server_key_iv: "718c1c145d5151e3cdd20d173f3632b8"
server_key_version: "encryptionkey@0"
I20260808 00:46:46.990528 18370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "9828a9886f044a8c8ec7f4f2ecf29876"
format_stamp: "Formatted at 2026-08-08 00:46:46 on dist-test-slave-kvfs"
server_key: "f276913e1c3f6f27454f9df6e83b1d70"
server_key_iv: "718c1c145d5151e3cdd20d173f3632b8"
server_key_version: "encryptionkey@0"
I20260808 00:46:46.992116 18370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:46:46.993039 18397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:46.993309 18370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:46.993453 18370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "9828a9886f044a8c8ec7f4f2ecf29876"
format_stamp: "Formatted at 2026-08-08 00:46:46 on dist-test-slave-kvfs"
server_key: "f276913e1c3f6f27454f9df6e83b1d70"
server_key_iv: "718c1c145d5151e3cdd20d173f3632b8"
server_key_version: "encryptionkey@0"
I20260808 00:46:46.993549 18370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:47.013353 18370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:47.016237 18370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:47.016438 18370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:47.021656 18370 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34059
I20260808 00:46:47.021713 18469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34059 every 8 connection(s)
I20260808 00:46:47.022169 18370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:47.023365 18471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:47.026559 18471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876: Bootstrap starting.
I20260808 00:46:47.026880 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 18370
I20260808 00:46:47.027064 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:47.027323 20901 external_mini_cluster.cc:1428] Setting key d85cbb143615450d6f65b7dcc211375a
I20260808 00:46:47.027627 18471 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:47.028151 18471 log.cc:824] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:47.029327 18471 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876: No bootstrap required, opened a new log
I20260808 00:46:47.032121 18471 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9828a9886f044a8c8ec7f4f2ecf29876" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34059 } }
I20260808 00:46:47.032312 18471 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:47.032341 18471 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9828a9886f044a8c8ec7f4f2ecf29876, State: Initialized, Role: FOLLOWER
I20260808 00:46:47.032541 18471 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9828a9886f044a8c8ec7f4f2ecf29876" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34059 } }
I20260808 00:46:47.032614 18471 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:47.032665 18471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:47.032724 18471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:47.033766 18471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9828a9886f044a8c8ec7f4f2ecf29876" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34059 } }
I20260808 00:46:47.033912 18471 leader_election.cc:302] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9828a9886f044a8c8ec7f4f2ecf29876; no voters:
I20260808 00:46:47.034139 18471 leader_election.cc:288] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:47.034192 18480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:47.034385 18480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [term 1 LEADER]: Becoming Leader. State: Replica: 9828a9886f044a8c8ec7f4f2ecf29876, State: Running, Role: LEADER
I20260808 00:46:47.034530 18471 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:47.034533 18480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9828a9886f044a8c8ec7f4f2ecf29876" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34059 } }
I20260808 00:46:47.034809 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:47.028122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46417 (local address 127.0.0.1:34059)
0808 00:46:47.028410 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:47.028421 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:47.028462 (+ 41us) server_negotiation.cc:407] Connection header received
0808 00:46:47.029030 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:47.029039 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:47.029095 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:47.029231 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:47.029586 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:47.030602 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:47.032000 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:47.032731 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:47.032844 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:47.033713 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:47.033718 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:47.033718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:47.034033 (+ 315us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:47.034088 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:47.034103 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:47.034496 (+ 393us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:47.034551 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 00:46:47.034582 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":127,"threads_started":1}
I20260808 00:46:47.034991 18480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9828a9886f044a8c8ec7f4f2ecf29876. Latest consensus state: current_term: 1 leader_uuid: "9828a9886f044a8c8ec7f4f2ecf29876" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9828a9886f044a8c8ec7f4f2ecf29876" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34059 } } }
I20260808 00:46:47.035068 18480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:47.035178 18481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9828a9886f044a8c8ec7f4f2ecf29876" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9828a9886f044a8c8ec7f4f2ecf29876" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34059 } } }
I20260808 00:46:47.035297 18486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:47.035367 18481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:47.036003 18486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:47.036091 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:46:47.038800 18486 catalog_manager.cc:1347] Generated new cluster ID: 09be663a22c146f88c1ffb8ad116fd67
I20260808 00:46:47.038870 18486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:47.042441 18477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:47.038151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37011 (local address 127.0.0.1:34059)
0808 00:46:47.038196 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:47.038198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:47.038410 (+ 212us) server_negotiation.cc:407] Connection header received
0808 00:46:47.038493 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:47.038494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:47.038507 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:47.038571 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:47.038755 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:47.039622 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:47.040832 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:47.041163 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:47.041250 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:47.042133 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:47.042134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:47.042134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:47.042140 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:47.042160 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:47.042166 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:47.042235 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:47.042320 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 00:46:47.042337 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:46:47.052639 18486 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:47.053354 18486 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:47.064527 18486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9828a9886f044a8c8ec7f4f2ecf29876: Generated new TSK 0
I20260808 00:46:47.064842 18486 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:47.070928 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 18370
W20260808 00:46:47.093425 18352 connection.cc:535] client connection to 127.0.0.1:34059 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34059): Connection reset by peer (error 104)
2026-08-08T00:46:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1629 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:47 dist-test-slave-kvfs krb5kdc[18520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:47 dist-test-slave-kvfs krb5kdc[18520](info): set up 2 sockets
Aug 08 00:46:47 dist-test-slave-kvfs krb5kdc[18520](info): commencing operation
krb5kdc: starting...
W20260808 00:46:48.358474 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:48 dist-test-slave-kvfs krb5kdc[18520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:48.511569 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:48.608474 18546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:48.608775 18546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:48.608814 18546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:48.610119 18546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:48.610195 18546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:48.610219 18546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:48.610240 18546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:48.610273 18546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:48.611961 18546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44237
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:48.612340 18546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:48.612627 18546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:48.619416 18554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:48.619416 18551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:48.619464 18552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:48.619587 18546 server_base.cc:1034] running on GCE node
I20260808 00:46:48.620004 18546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:48.620397 18546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:48.621568 18546 hybrid_clock.cc:648] HybridClock initialized: now 1786150008621547 us; error 43 us; skew 500 ppm
Aug 08 00:46:48 dist-test-slave-kvfs krb5kdc[18520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150008, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:48.624667 18546 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:48.625207 18546 webserver.cc:460] Webserver started at http://127.0.0.1:34075/ using document root <none> and password file <none>
I20260808 00:46:48.625501 18546 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:48.625578 18546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:48.625751 18546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:48.627049 18546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "02f43fb72a0d45429f7535f80867d85d"
format_stamp: "Formatted at 2026-08-08 00:46:48 on dist-test-slave-kvfs"
server_key: "f6203843bb5f083ae4b64cdfca0d5815"
server_key_iv: "f5b752a106d9cb278f2b7f8d02f08d37"
server_key_version: "encryptionkey@0"
I20260808 00:46:48.627485 18546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "02f43fb72a0d45429f7535f80867d85d"
format_stamp: "Formatted at 2026-08-08 00:46:48 on dist-test-slave-kvfs"
server_key: "f6203843bb5f083ae4b64cdfca0d5815"
server_key_iv: "f5b752a106d9cb278f2b7f8d02f08d37"
server_key_version: "encryptionkey@0"
I20260808 00:46:48.629194 18546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 00:46:48.630131 18564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:48.630373 18546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:46:48.630479 18546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "02f43fb72a0d45429f7535f80867d85d"
format_stamp: "Formatted at 2026-08-08 00:46:48 on dist-test-slave-kvfs"
server_key: "f6203843bb5f083ae4b64cdfca0d5815"
server_key_iv: "f5b752a106d9cb278f2b7f8d02f08d37"
server_key_version: "encryptionkey@0"
I20260808 00:46:48.630574 18546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:48.634568 18546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:48.636898 18546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:48.637105 18546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:48.642274 18546 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44237
I20260808 00:46:48.642343 18645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44237 every 8 connection(s)
I20260808 00:46:48.642773 18546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:48.644148 18646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:48.647197 18646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d: Bootstrap starting.
I20260808 00:46:48.647985 18646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:48.648417 18646 log.cc:824] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:48.649389 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 18546
I20260808 00:46:48.649439 18646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d: No bootstrap required, opened a new log
I20260808 00:46:48.649588 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:48.649835 20901 external_mini_cluster.cc:1428] Setting key dc0a126991752210ce9c66f5e027723f
I20260808 00:46:48.652129 18646 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f43fb72a0d45429f7535f80867d85d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } }
I20260808 00:46:48.652377 18646 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:48.652489 18646 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f43fb72a0d45429f7535f80867d85d, State: Initialized, Role: FOLLOWER
I20260808 00:46:48.653120 18646 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f43fb72a0d45429f7535f80867d85d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } }
I20260808 00:46:48.653280 18646 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:48.653314 18646 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:48.653414 18646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:48.654071 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:48.650558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52421 (local address 127.0.0.1:44237)
0808 00:46:48.650766 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:48.650776 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:48.650806 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:46:48.652070 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:48.652091 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:48.652131 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:48.652284 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:48.653225 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:48.653230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:48.653230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:48.653235 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:48.653317 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:48.653344 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:48.653443 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:48.653541 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 00:46:48.653593 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260808 00:46:48.654354 18646 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f43fb72a0d45429f7535f80867d85d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } }
I20260808 00:46:48.654505 18646 leader_election.cc:302] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02f43fb72a0d45429f7535f80867d85d; no voters:
I20260808 00:46:48.654706 18646 leader_election.cc:288] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:48.654724 18651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:48.655082 18646 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:48.655130 18651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [term 1 LEADER]: Becoming Leader. State: Replica: 02f43fb72a0d45429f7535f80867d85d, State: Running, Role: LEADER
I20260808 00:46:48.655270 18651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f43fb72a0d45429f7535f80867d85d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } }
I20260808 00:46:48.655563 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 00:46:48.658206 18678 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:46:48.658344 18678 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:46:48.658286 18652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02f43fb72a0d45429f7535f80867d85d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f43fb72a0d45429f7535f80867d85d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } } }
I20260808 00:46:48.658476 18652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:48.658802 18653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02f43fb72a0d45429f7535f80867d85d. Latest consensus state: current_term: 1 leader_uuid: "02f43fb72a0d45429f7535f80867d85d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f43fb72a0d45429f7535f80867d85d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44237 } } }
I20260808 00:46:48.658978 18653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:48.659001 18681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:48.659318 18681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:48.659394 18650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:48.657983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49371 (local address 127.0.0.1:44237)
0808 00:46:48.658039 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:48.658041 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:48.658052 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:46:48.658167 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:48.658168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:48.658173 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:48.658206 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:48.659074 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:48.659076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:48.659076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:48.659081 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:48.659099 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:48.659104 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:48.659135 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:48.659261 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 00:46:48.659282 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:46:48.661368 18681 catalog_manager.cc:1347] Generated new cluster ID: 689712a7d1b44212950b89df03e2fcc2
I20260808 00:46:48.661435 18681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:48.666114 18681 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:48.666270 18681 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:48.670194 18681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02f43fb72a0d45429f7535f80867d85d: Generated new TSK 0
I20260808 00:46:48.670375 18681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:48.674106 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 18546
2026-08-08T00:46:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:48 dist-test-slave-kvfs krb5kdc[18688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:48 dist-test-slave-kvfs krb5kdc[18688](info): set up 2 sockets
Aug 08 00:46:48 dist-test-slave-kvfs krb5kdc[18688](info): commencing operation
krb5kdc: starting...
W20260808 00:46:49.947319 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:50 dist-test-slave-kvfs krb5kdc[18688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:50.101058 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:50.199352 18712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:50.199683 18712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:50.199726 18712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:50.201054 18712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:50.201143 18712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:50.201167 18712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:50.201190 18712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:50.201217 18712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:50.202955 18712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43189
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:50.203307 18712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:50.203598 18712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:50.209560 18723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:50.209560 18727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:50.209607 18722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:50.209968 18712 server_base.cc:1034] running on GCE node
I20260808 00:46:50.210310 18712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:50.210626 18712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:50.211808 18712 hybrid_clock.cc:648] HybridClock initialized: now 1786150010211787 us; error 35 us; skew 500 ppm
Aug 08 00:46:50 dist-test-slave-kvfs krb5kdc[18688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150010, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:50.215039 18712 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:50.215549 18712 webserver.cc:460] Webserver started at http://127.0.0.1:36335/ using document root <none> and password file <none>
I20260808 00:46:50.215822 18712 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:50.215906 18712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:50.216133 18712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:50.217437 18712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "bd9fb93d012043948b644e95b5906043"
format_stamp: "Formatted at 2026-08-08 00:46:50 on dist-test-slave-kvfs"
server_key: "9f734a3e43210600281f7c0f5b648e07"
server_key_iv: "9e0256f1e22d2bb7fb86c34853865729"
server_key_version: "encryptionkey@0"
I20260808 00:46:50.217885 18712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "bd9fb93d012043948b644e95b5906043"
format_stamp: "Formatted at 2026-08-08 00:46:50 on dist-test-slave-kvfs"
server_key: "9f734a3e43210600281f7c0f5b648e07"
server_key_iv: "9e0256f1e22d2bb7fb86c34853865729"
server_key_version: "encryptionkey@0"
I20260808 00:46:50.219517 18712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:46:50.220556 18737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:50.220810 18712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:46:50.220981 18712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "bd9fb93d012043948b644e95b5906043"
format_stamp: "Formatted at 2026-08-08 00:46:50 on dist-test-slave-kvfs"
server_key: "9f734a3e43210600281f7c0f5b648e07"
server_key_iv: "9e0256f1e22d2bb7fb86c34853865729"
server_key_version: "encryptionkey@0"
I20260808 00:46:50.221082 18712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:50.248262 18712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:50.251132 18712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:50.251338 18712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:50.256763 18712 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43189
I20260808 00:46:50.256829 18812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43189 every 8 connection(s)
I20260808 00:46:50.257323 18712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:50.258613 18813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:50.259964 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 18712
I20260808 00:46:50.260172 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:50.260430 20901 external_mini_cluster.cc:1428] Setting key b5596014690b2c2a02355625714ea42d
I20260808 00:46:50.262449 18813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043: Bootstrap starting.
I20260808 00:46:50.263537 18813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:50.263950 18813 log.cc:824] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:50.264966 18813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043: No bootstrap required, opened a new log
I20260808 00:46:50.268285 18813 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9fb93d012043948b644e95b5906043" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } }
I20260808 00:46:50.268554 18813 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:50.268636 18813 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd9fb93d012043948b644e95b5906043, State: Initialized, Role: FOLLOWER
I20260808 00:46:50.268832 18813 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9fb93d012043948b644e95b5906043" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } }
I20260808 00:46:50.268915 18813 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:50.268965 18813 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:50.269024 18813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 00:46:50 dist-test-slave-kvfs krb5kdc[18688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:46:50.269881 18813 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9fb93d012043948b644e95b5906043" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } }
I20260808 00:46:50.270013 18813 leader_election.cc:302] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd9fb93d012043948b644e95b5906043; no voters:
I20260808 00:46:50.270253 18813 leader_election.cc:288] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:50.270328 18819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:50.270514 18819 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [term 1 LEADER]: Becoming Leader. State: Replica: bd9fb93d012043948b644e95b5906043, State: Running, Role: LEADER
I20260808 00:46:50.270629 18819 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9fb93d012043948b644e95b5906043" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } }
I20260808 00:46:50.270664 18813 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:50.271049 18819 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd9fb93d012043948b644e95b5906043. Latest consensus state: current_term: 1 leader_uuid: "bd9fb93d012043948b644e95b5906043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9fb93d012043948b644e95b5906043" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } } }
I20260808 00:46:50.271159 18819 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:50.271134 18820 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd9fb93d012043948b644e95b5906043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9fb93d012043948b644e95b5906043" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43189 } } }
I20260808 00:46:50.271478 18820 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:50.271519 18826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:50.272159 18826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:50.274295 18826 catalog_manager.cc:1347] Generated new cluster ID: f7218cd8f724451f94a760da0a375b41
I20260808 00:46:50.274346 18826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:50.274736 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:50.261153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54353 (local address 127.0.0.1:43189)
0808 00:46:50.261353 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:50.261363 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:50.261395 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:46:50.262024 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:50.262032 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:50.262092 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:50.262236 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:50.263364 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:50.264575 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:50.265778 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:50.266247 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:50.266334 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:50.269925 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:50.269930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:50.269931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:50.269947 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:50.272766 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:50.273238 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:50.273242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:50.273243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:50.273357 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:50.273609 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:50.273610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:50.273611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:50.273992 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:50.274178 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:50.274326 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 00:46:50.274492 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260808 00:46:50.275810 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:46:50.287706 18826 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:50.288657 18826 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:50.288744 18816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:50.277757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52711 (local address 127.0.0.1:43189)
0808 00:46:50.277800 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:50.277802 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:50.280200 (+ 2398us) server_negotiation.cc:407] Connection header received
0808 00:46:50.280271 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:50.280272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:50.280285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:50.280358 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:50.281033 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:50.281998 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:50.283042 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:50.283417 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:50.283497 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:50.285132 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:50.285134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:50.285134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:50.285142 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:50.287006 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:50.287437 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:50.287439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:50.287439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:50.287512 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:50.287866 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:50.287867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:50.287868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:50.288007 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:50.288158 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:50.288383 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 00:46:50.288577 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":21}
I20260808 00:46:50.295352 18826 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd9fb93d012043948b644e95b5906043: Generated new TSK 0
I20260808 00:46:50.295578 18826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:50.302935 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 18712
2026-08-08T00:46:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:50 dist-test-slave-kvfs krb5kdc[18857](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:50 dist-test-slave-kvfs krb5kdc[18857](info): set up 2 sockets
Aug 08 00:46:50 dist-test-slave-kvfs krb5kdc[18857](info): commencing operation
krb5kdc: starting...
W20260808 00:46:51.584216 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:51 dist-test-slave-kvfs krb5kdc[18857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:51.736788 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34915
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:51.834209 18878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:51.834583 18878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:51.834627 18878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:51.837163 18878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:51.837262 18878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:51.837287 18878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:51.837311 18878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:51.837330 18878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:51.840436 18878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:51.840866 18878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:51.841244 18878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:51.852442 18889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:51.852443 18885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:51.852468 18886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:51.853765 18878 server_base.cc:1034] running on GCE node
I20260808 00:46:51.854161 18878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:51.854629 18878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:51.855842 18878 hybrid_clock.cc:648] HybridClock initialized: now 1786150011855809 us; error 50 us; skew 500 ppm
Aug 08 00:46:51 dist-test-slave-kvfs krb5kdc[18857](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150011, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:51.860427 18878 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:51.865087 18878 webserver.cc:460] Webserver started at http://127.0.0.1:36141/ using document root <none> and password file <none>
I20260808 00:46:51.865480 18878 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:51.865563 18878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:51.865737 18878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:51.867600 18878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "362c086533fe439688fe4bf53a276ac6"
format_stamp: "Formatted at 2026-08-08 00:46:51 on dist-test-slave-kvfs"
server_key: "4a91ed8f971c32fbaf9a1d01ce461dc2"
server_key_iv: "1e909133255c5170b2b844cf4e6b8813"
server_key_version: "encryptionkey@0"
I20260808 00:46:51.868129 18878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "362c086533fe439688fe4bf53a276ac6"
format_stamp: "Formatted at 2026-08-08 00:46:51 on dist-test-slave-kvfs"
server_key: "4a91ed8f971c32fbaf9a1d01ce461dc2"
server_key_iv: "1e909133255c5170b2b844cf4e6b8813"
server_key_version: "encryptionkey@0"
I20260808 00:46:51.870682 18878 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:46:51.872088 18900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:51.872313 18878 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:51.872444 18878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "362c086533fe439688fe4bf53a276ac6"
format_stamp: "Formatted at 2026-08-08 00:46:51 on dist-test-slave-kvfs"
server_key: "4a91ed8f971c32fbaf9a1d01ce461dc2"
server_key_iv: "1e909133255c5170b2b844cf4e6b8813"
server_key_version: "encryptionkey@0"
I20260808 00:46:51.872543 18878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:51.891009 18878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:51.894373 18878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:51.894595 18878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:51.901100 18878 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34915
I20260808 00:46:51.901151 18975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34915 every 8 connection(s)
I20260808 00:46:51.901659 18878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:51.903105 18976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:51.906234 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 18878
I20260808 00:46:51.906414 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:51.906656 20901 external_mini_cluster.cc:1428] Setting key 60bbc7a5bd3618d185b0372be46c37e8
I20260808 00:46:51.907640 18976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6: Bootstrap starting.
I20260808 00:46:51.908509 18976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:51.908874 18976 log.cc:824] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:51.909736 18976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6: No bootstrap required, opened a new log
I20260808 00:46:51.912155 18976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c086533fe439688fe4bf53a276ac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34915 } }
I20260808 00:46:51.912345 18976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:51.912397 18976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362c086533fe439688fe4bf53a276ac6, State: Initialized, Role: FOLLOWER
I20260808 00:46:51.912564 18976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c086533fe439688fe4bf53a276ac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34915 } }
I20260808 00:46:51.912669 18976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:51.912717 18976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:51.912765 18976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:51.913476 18976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c086533fe439688fe4bf53a276ac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34915 } }
I20260808 00:46:51.913566 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:51.907402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41753 (local address 127.0.0.1:34915)
0808 00:46:51.907656 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:51.907666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:51.907697 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:46:51.908383 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:51.908393 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:51.908454 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:51.908657 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:51.908947 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:51.910293 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:51.911296 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:51.911746 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:51.911921 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:51.912710 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:51.912715 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:51.912716 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:51.912991 (+ 275us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:51.913029 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:51.913042 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:51.913116 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:51.913206 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 00:46:51.913291 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":130,"threads_started":1}
I20260808 00:46:51.913602 18976 leader_election.cc:302] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 362c086533fe439688fe4bf53a276ac6; no voters:
I20260808 00:46:51.913775 18976 leader_election.cc:288] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:51.913833 18984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:51.914021 18984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [term 1 LEADER]: Becoming Leader. State: Replica: 362c086533fe439688fe4bf53a276ac6, State: Running, Role: LEADER
I20260808 00:46:51.914081 18976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:51.914139 18984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c086533fe439688fe4bf53a276ac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34915 } }
I20260808 00:46:51.914556 18984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 362c086533fe439688fe4bf53a276ac6. Latest consensus state: current_term: 1 leader_uuid: "362c086533fe439688fe4bf53a276ac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c086533fe439688fe4bf53a276ac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34915 } } }
I20260808 00:46:51.914620 18984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:51.914662 18985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "362c086533fe439688fe4bf53a276ac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "362c086533fe439688fe4bf53a276ac6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34915 } } }
I20260808 00:46:51.914758 18985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:51.914777 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 00:46:51.916275 19004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:46:51.916329 19004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:46:51.917161 18993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:51.917423 18993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:51.919375 18993 catalog_manager.cc:1347] Generated new cluster ID: 0d6ac3e217854322aaa34f4a76d3bb13
I20260808 00:46:51.919430 18993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:51.920709 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:51.916732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55835 (local address 127.0.0.1:34915)
0808 00:46:51.916777 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:51.916780 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:51.916887 (+ 107us) server_negotiation.cc:407] Connection header received
0808 00:46:51.916944 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:51.916944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:51.916957 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:51.917013 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:51.917235 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:51.917944 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:51.919281 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:51.919573 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:51.919621 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:51.920455 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:51.920456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:51.920457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:51.920461 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:51.920477 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:51.920481 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:51.920509 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:51.920605 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 00:46:51.920632 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:46:51.959964 18993 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:51.960726 18993 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:51.970791 18993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 362c086533fe439688fe4bf53a276ac6: Generated new TSK 0
I20260808 00:46:51.971107 18993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:51.975008 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 18878
2026-08-08T00:46:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1665 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:52 dist-test-slave-kvfs krb5kdc[19018](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:52 dist-test-slave-kvfs krb5kdc[19018](info): set up 2 sockets
Aug 08 00:46:52 dist-test-slave-kvfs krb5kdc[19018](info): commencing operation
krb5kdc: starting...
W20260808 00:46:53.326890 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.327s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:53 dist-test-slave-kvfs krb5kdc[19018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:53.477627 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34355
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:53.574563 19037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:53.574815 19037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:53.574839 19037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:53.575976 19037 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:53.576009 19037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:53.576087 19037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:53.576109 19037 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:53.576133 19037 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:53.577683 19037 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:53.577934 19037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:53.578133 19037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:53.583698 19051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:53.583698 19047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:53.583758 19044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:53.583971 19037 server_base.cc:1034] running on GCE node
I20260808 00:46:53.584398 19037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:53.584760 19037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:53.585929 19037 hybrid_clock.cc:648] HybridClock initialized: now 1786150013585930 us; error 45 us; skew 500 ppm
Aug 08 00:46:53 dist-test-slave-kvfs krb5kdc[19018](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150013, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:53.589075 19037 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:53.589522 19037 webserver.cc:460] Webserver started at http://127.0.0.1:45301/ using document root <none> and password file <none>
I20260808 00:46:53.589800 19037 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:53.589880 19037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:53.590045 19037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:53.591267 19037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "fb8b91de3c6b411c840f8ca251a5402f"
format_stamp: "Formatted at 2026-08-08 00:46:53 on dist-test-slave-kvfs"
server_key: "11b68c7ac525d701a6cd95bac708b45f"
server_key_iv: "5f83b871da20220a3846eb8df6842f44"
server_key_version: "encryptionkey@0"
I20260808 00:46:53.591693 19037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "fb8b91de3c6b411c840f8ca251a5402f"
format_stamp: "Formatted at 2026-08-08 00:46:53 on dist-test-slave-kvfs"
server_key: "11b68c7ac525d701a6cd95bac708b45f"
server_key_iv: "5f83b871da20220a3846eb8df6842f44"
server_key_version: "encryptionkey@0"
I20260808 00:46:53.593312 19037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 00:46:53.594296 19060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:53.594540 19037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:53.594679 19037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "fb8b91de3c6b411c840f8ca251a5402f"
format_stamp: "Formatted at 2026-08-08 00:46:53 on dist-test-slave-kvfs"
server_key: "11b68c7ac525d701a6cd95bac708b45f"
server_key_iv: "5f83b871da20220a3846eb8df6842f44"
server_key_version: "encryptionkey@0"
I20260808 00:46:53.594808 19037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:53.599081 19037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:53.602497 19037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:53.602706 19037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:53.608579 19037 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34355
I20260808 00:46:53.608613 19140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34355 every 8 connection(s)
I20260808 00:46:53.609081 19037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:53.610420 19141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:53.613289 19141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f: Bootstrap starting.
I20260808 00:46:53.614027 19141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:53.614359 19141 log.cc:824] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:53.615162 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 19037
I20260808 00:46:53.615355 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:53.615366 19141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f: No bootstrap required, opened a new log
I20260808 00:46:53.615573 20901 external_mini_cluster.cc:1428] Setting key 3b9ca650ef0ffd2b8ce7bf90ed229e75
I20260808 00:46:53.617673 19141 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8b91de3c6b411c840f8ca251a5402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34355 } }
I20260808 00:46:53.617832 19141 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:53.617856 19141 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8b91de3c6b411c840f8ca251a5402f, State: Initialized, Role: FOLLOWER
I20260808 00:46:53.618034 19141 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8b91de3c6b411c840f8ca251a5402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34355 } }
I20260808 00:46:53.618145 19141 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:53.618182 19141 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:53.618242 19141 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:53.619338 19145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:53.616379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35009 (local address 127.0.0.1:34355)
0808 00:46:53.616601 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:53.616611 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:53.616645 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:46:53.617673 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:53.617683 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:53.617727 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:53.617974 (+ 247us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:53.618768 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:53.618772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:53.618772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:53.618775 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:53.618823 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:53.618836 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:53.618895 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:53.619070 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 00:46:53.619128 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":116,"threads_started":1}
I20260808 00:46:53.619441 19141 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8b91de3c6b411c840f8ca251a5402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34355 } }
I20260808 00:46:53.619567 19141 leader_election.cc:302] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb8b91de3c6b411c840f8ca251a5402f; no voters:
I20260808 00:46:53.619766 19141 leader_election.cc:288] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:53.619807 19146 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:53.619946 19146 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [term 1 LEADER]: Becoming Leader. State: Replica: fb8b91de3c6b411c840f8ca251a5402f, State: Running, Role: LEADER
I20260808 00:46:53.620087 19141 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:53.620054 19146 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8b91de3c6b411c840f8ca251a5402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34355 } }
I20260808 00:46:53.620534 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:46:53.620534 19148 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb8b91de3c6b411c840f8ca251a5402f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8b91de3c6b411c840f8ca251a5402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34355 } } }
I20260808 00:46:53.620553 19147 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb8b91de3c6b411c840f8ca251a5402f. Latest consensus state: current_term: 1 leader_uuid: "fb8b91de3c6b411c840f8ca251a5402f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8b91de3c6b411c840f8ca251a5402f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34355 } } }
I20260808 00:46:53.620630 19148 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:53.620682 19147 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:53.621886 19155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:53.622576 19155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:53.624271 19155 catalog_manager.cc:1347] Generated new cluster ID: cad739b81c854b37a0572600f9f9a2ea
I20260808 00:46:53.624343 19155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:53.629316 19145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:53.622562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34251 (local address 127.0.0.1:34355)
0808 00:46:53.622598 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:53.622600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:53.628087 (+ 5487us) server_negotiation.cc:407] Connection header received
0808 00:46:53.628222 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:53.628223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:53.628242 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:53.628294 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:53.629063 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:53.629064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:53.629065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:53.629068 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:53.629086 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:53.629091 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:53.629145 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:53.629231 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 00:46:53.629247 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:46:53.632979 19155 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:53.633179 19155 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:53.644217 19155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb8b91de3c6b411c840f8ca251a5402f: Generated new TSK 0
I20260808 00:46:53.644486 19155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:53.648108 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 19037
2026-08-08T00:46:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1682 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:53 dist-test-slave-kvfs krb5kdc[19178](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:53 dist-test-slave-kvfs krb5kdc[19178](info): set up 2 sockets
Aug 08 00:46:53 dist-test-slave-kvfs krb5kdc[19178](info): commencing operation
krb5kdc: starting...
W20260808 00:46:54.995223 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:55 dist-test-slave-kvfs krb5kdc[19178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:55.156877 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:55.260540 19195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:55.260845 19195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:55.260892 19195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:55.262176 19195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:55.262252 19195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:55.262277 19195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:55.262311 19195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:55.262339 19195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:55.264577 19195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:55.264989 19195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:55.265313 19195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:55.271550 19204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:55.271685 19195 server_base.cc:1034] running on GCE node
W20260808 00:46:55.271518 19206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:55.271546 19203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:55.272192 19195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:55.272586 19195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:55.273752 19195 hybrid_clock.cc:648] HybridClock initialized: now 1786150015273729 us; error 42 us; skew 500 ppm
Aug 08 00:46:55 dist-test-slave-kvfs krb5kdc[19178](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150015, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:55.276939 19195 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:55.277545 19195 webserver.cc:460] Webserver started at http://127.0.0.1:41555/ using document root <none> and password file <none>
I20260808 00:46:55.277848 19195 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:55.277901 19195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:55.278056 19195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:55.279349 19195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "66e150ec81f74195bff4c55c2a57f9a2"
format_stamp: "Formatted at 2026-08-08 00:46:55 on dist-test-slave-kvfs"
server_key: "4a921679079fe0682c2e5f270565193d"
server_key_iv: "e0689887195cab8529732dff7616469a"
server_key_version: "encryptionkey@0"
I20260808 00:46:55.279784 19195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "66e150ec81f74195bff4c55c2a57f9a2"
format_stamp: "Formatted at 2026-08-08 00:46:55 on dist-test-slave-kvfs"
server_key: "4a921679079fe0682c2e5f270565193d"
server_key_iv: "e0689887195cab8529732dff7616469a"
server_key_version: "encryptionkey@0"
I20260808 00:46:55.281425 19195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:46:55.282467 19218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:55.282724 19195 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:55.282838 19195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "66e150ec81f74195bff4c55c2a57f9a2"
format_stamp: "Formatted at 2026-08-08 00:46:55 on dist-test-slave-kvfs"
server_key: "4a921679079fe0682c2e5f270565193d"
server_key_iv: "e0689887195cab8529732dff7616469a"
server_key_version: "encryptionkey@0"
I20260808 00:46:55.282943 19195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:55.320705 19195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:55.323514 19195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:55.323781 19195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:55.329404 19195 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34961
I20260808 00:46:55.329461 19291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34961 every 8 connection(s)
I20260808 00:46:55.330004 19195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:55.331246 19292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:55.335060 19292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2: Bootstrap starting.
I20260808 00:46:55.335853 19292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:55.336282 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 19195
I20260808 00:46:55.336310 19292 log.cc:824] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:55.336441 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:55.336697 20901 external_mini_cluster.cc:1428] Setting key 60b83c532db5ca420604750d2f4f3317
I20260808 00:46:55.337522 19292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2: No bootstrap required, opened a new log
I20260808 00:46:55.340302 19292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e150ec81f74195bff4c55c2a57f9a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34961 } }
I20260808 00:46:55.340520 19292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:55.340567 19292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66e150ec81f74195bff4c55c2a57f9a2, State: Initialized, Role: FOLLOWER
I20260808 00:46:55.340725 19292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e150ec81f74195bff4c55c2a57f9a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34961 } }
I20260808 00:46:55.340850 19292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:55.340902 19292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:55.340962 19292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:55.341964 19292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e150ec81f74195bff4c55c2a57f9a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34961 } }
I20260808 00:46:55.342100 19292 leader_election.cc:302] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 66e150ec81f74195bff4c55c2a57f9a2; no voters:
I20260808 00:46:55.342298 19292 leader_election.cc:288] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:55.342366 19304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:55.342597 19304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [term 1 LEADER]: Becoming Leader. State: Replica: 66e150ec81f74195bff4c55c2a57f9a2, State: Running, Role: LEADER
I20260808 00:46:55.342623 19292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:55.342703 19304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e150ec81f74195bff4c55c2a57f9a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34961 } }
I20260808 00:46:55.343235 19305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "66e150ec81f74195bff4c55c2a57f9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e150ec81f74195bff4c55c2a57f9a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34961 } } }
I20260808 00:46:55.343237 19307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 66e150ec81f74195bff4c55c2a57f9a2. Latest consensus state: current_term: 1 leader_uuid: "66e150ec81f74195bff4c55c2a57f9a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "66e150ec81f74195bff4c55c2a57f9a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34961 } } }
I20260808 00:46:55.343370 19305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:55.343379 19307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:55.344185 19318 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:55.344837 19318 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:55.347868 19318 catalog_manager.cc:1347] Generated new cluster ID: 6227321e57e44835a2aab0572fc4ae92
I20260808 00:46:55.347935 19318 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:46:55 dist-test-slave-kvfs krb5kdc[19178](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:46:55.354038 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:55.337577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:41825 (local address 127.0.0.1:34961)
0808 00:46:55.337826 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:55.337838 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:55.337901 (+ 63us) server_negotiation.cc:407] Connection header received
0808 00:46:55.339239 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:55.339253 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:55.339343 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:55.339556 (+ 213us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:55.340565 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:55.342765 (+ 2200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:55.344325 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:55.345245 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:55.345379 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:55.349165 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:55.349171 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:55.349171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:55.349187 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:55.351540 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:55.352250 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:55.352255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:55.352255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:55.352411 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:55.352779 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:55.352780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:55.352781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:55.353211 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:55.353421 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:55.353630 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 00:46:55.353761 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":122,"threads_started":1}
I20260808 00:46:55.355285 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:46:55.366979 19301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:55.357457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:43967 (local address 127.0.0.1:34961)
0808 00:46:55.357502 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:55.357504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:55.357564 (+ 60us) server_negotiation.cc:407] Connection header received
0808 00:46:55.357736 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:55.357738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:55.357753 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:55.357835 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:55.358631 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:55.359821 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:55.361140 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:55.361550 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:55.361633 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:55.363572 (+ 1939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:55.363573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:55.363574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:46:55.363584 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:55.365548 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:55.366039 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:55.366041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:55.366041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:55.366118 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:46:55.366362 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:46:55.366363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:46:55.366363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:46:55.366533 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:55.366655 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:55.366774 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 00:46:55.366852 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:46:55.370728 19318 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:55.371358 19318 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:55.377336 19318 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 66e150ec81f74195bff4c55c2a57f9a2: Generated new TSK 0
I20260808 00:46:55.377658 19318 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:55.385003 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 19195
W20260808 00:46:55.402060 19176 connection.cc:535] client connection to 127.0.0.1:34961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34961): Connection reset by peer (error 104)
2026-08-08T00:46:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1733 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:55 dist-test-slave-kvfs krb5kdc[19337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:55 dist-test-slave-kvfs krb5kdc[19337](info): set up 2 sockets
Aug 08 00:46:55 dist-test-slave-kvfs krb5kdc[19337](info): commencing operation
krb5kdc: starting...
W20260808 00:46:56.679975 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:56 dist-test-slave-kvfs krb5kdc[19337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:56.836604 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45901
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40217
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:56.935338 19358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:56.935685 19358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:56.935729 19358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:56.937114 19358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:56.937207 19358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:56.937232 19358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:56.937255 19358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:56.937288 19358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:56.939087 19358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40217
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:56.939467 19358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:56.939785 19358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:56.945933 19368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:56.946013 19366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:56.945937 19365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:56.946225 19358 server_base.cc:1034] running on GCE node
I20260808 00:46:56.946594 19358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:56.947031 19358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:56.948278 19358 hybrid_clock.cc:648] HybridClock initialized: now 1786150016948240 us; error 56 us; skew 500 ppm
Aug 08 00:46:56 dist-test-slave-kvfs krb5kdc[19337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150016, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:56.951686 19358 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:56.952379 19358 webserver.cc:460] Webserver started at http://127.0.0.1:46461/ using document root <none> and password file <none>
I20260808 00:46:56.952693 19358 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:56.952745 19358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:56.952903 19358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:56.955049 19358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "03a3198062374bcba030530b8d1144a3"
format_stamp: "Formatted at 2026-08-08 00:46:56 on dist-test-slave-kvfs"
server_key: "c57d1374aa2a287c1b3585ea26796541"
server_key_iv: "f47bcda59683ae04df35eb14ff50e054"
server_key_version: "encryptionkey@0"
I20260808 00:46:56.955648 19358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "03a3198062374bcba030530b8d1144a3"
format_stamp: "Formatted at 2026-08-08 00:46:56 on dist-test-slave-kvfs"
server_key: "c57d1374aa2a287c1b3585ea26796541"
server_key_iv: "f47bcda59683ae04df35eb14ff50e054"
server_key_version: "encryptionkey@0"
I20260808 00:46:56.958094 19358 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:46:56.959319 19377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:56.959542 19358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:56.959664 19358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "03a3198062374bcba030530b8d1144a3"
format_stamp: "Formatted at 2026-08-08 00:46:56 on dist-test-slave-kvfs"
server_key: "c57d1374aa2a287c1b3585ea26796541"
server_key_iv: "f47bcda59683ae04df35eb14ff50e054"
server_key_version: "encryptionkey@0"
I20260808 00:46:56.959765 19358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:56.975394 19358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:56.977911 19358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:56.978051 19358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:56.983376 19358 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45901
I20260808 00:46:56.983417 19453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45901 every 8 connection(s)
I20260808 00:46:56.983876 19358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:56.984874 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 19358
I20260808 00:46:56.985023 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:56.985162 19454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:56.985293 20901 external_mini_cluster.cc:1428] Setting key ef57395e80000256311fafc00c534f6b
I20260808 00:46:56.988685 19454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3: Bootstrap starting.
I20260808 00:46:56.989462 19454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:56.989789 19454 log.cc:824] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:56.990628 19454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3: No bootstrap required, opened a new log
I20260808 00:46:56.991714 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:56.985938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:56187 (local address 127.0.0.1:45901)
0808 00:46:56.986205 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:56.986214 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:56.986244 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:46:56.986901 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:56.986910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:56.986966 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:56.987141 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:56.987392 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:56.988494 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:56.989426 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:56.989980 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:56.990130 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:56.991005 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:56.991009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:56.991010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:56.991218 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:56.991256 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:56.991269 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:56.991369 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:56.991458 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 00:46:56.991492 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":126,"threads_started":1}
I20260808 00:46:56.993173 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:46:56.993336 19454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a3198062374bcba030530b8d1144a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } }
I20260808 00:46:56.993523 19454 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:56.993557 19454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a3198062374bcba030530b8d1144a3, State: Initialized, Role: FOLLOWER
I20260808 00:46:56.993688 19454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a3198062374bcba030530b8d1144a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } }
I20260808 00:46:56.993760 19454 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:56.993796 19454 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:56.993848 19454 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:56.994771 19454 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a3198062374bcba030530b8d1144a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } }
I20260808 00:46:56.994907 19454 leader_election.cc:302] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03a3198062374bcba030530b8d1144a3; no voters:
I20260808 00:46:56.995096 19454 leader_election.cc:288] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:56.995128 19468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:56.995252 19468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [term 1 LEADER]: Becoming Leader. State: Replica: 03a3198062374bcba030530b8d1144a3, State: Running, Role: LEADER
I20260808 00:46:56.995337 19468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a3198062374bcba030530b8d1144a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } }
I20260808 00:46:56.995404 19454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:56.995765 19468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03a3198062374bcba030530b8d1144a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a3198062374bcba030530b8d1144a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } } }
I20260808 00:46:56.995850 19468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:56.996001 19468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03a3198062374bcba030530b8d1144a3. Latest consensus state: current_term: 1 leader_uuid: "03a3198062374bcba030530b8d1144a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03a3198062374bcba030530b8d1144a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45901 } } }
I20260808 00:46:56.996104 19468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:56.996294 19475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:56.996986 19475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:56.999925 19475 catalog_manager.cc:1347] Generated new cluster ID: eb10f93836094f3db28608d29cdc076c
I20260808 00:46:56.999996 19475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:57.000458 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:56.995204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:54677 (local address 127.0.0.1:45901)
0808 00:46:56.995250 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:56.995252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:56.997141 (+ 1889us) server_negotiation.cc:407] Connection header received
0808 00:46:56.997233 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:56.997234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:56.997247 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:56.997302 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:56.997519 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:56.998037 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:56.998923 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:46:56.999214 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:46:56.999311 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:46:57.000138 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:57.000139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:57.000139 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:57.000144 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:57.000164 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:57.000169 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:57.000229 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:57.000367 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 00:46:57.000379 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:46:57.018373 19475 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:57.019101 19475 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:57.026739 19475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03a3198062374bcba030530b8d1144a3: Generated new TSK 0
I20260808 00:46:57.026995 19475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:57.032258 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 19358
W20260808 00:46:57.046092 19335 connection.cc:535] client connection to 127.0.0.1:45901 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45901): Connection reset by peer (error 104)
2026-08-08T00:46:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1643 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:57 dist-test-slave-kvfs krb5kdc[19498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:57 dist-test-slave-kvfs krb5kdc[19498](info): set up 2 sockets
Aug 08 00:46:57 dist-test-slave-kvfs krb5kdc[19498](info): commencing operation
krb5kdc: starting...
W20260808 00:46:58.405581 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.344s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:46:58 dist-test-slave-kvfs krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150018, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:46:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:46:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:46:58.560622 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39027
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:46:58.658977 19514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:46:58.659318 19514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:46:58.659365 19514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:46:58.660851 19514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:46:58.660949 19514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:46:58.660979 19514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:46:58.661002 19514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:46:58.661041 19514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:46:58.662876 19514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:46:58.663178 19514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:46:58.663415 19514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:46:58.669348 19527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:58.669348 19523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:46:58.669389 19524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:46:58.669883 19514 server_base.cc:1034] running on GCE node
I20260808 00:46:58.670230 19514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:46:58.670619 19514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:46:58.671825 19514 hybrid_clock.cc:648] HybridClock initialized: now 1786150018671776 us; error 69 us; skew 500 ppm
Aug 08 00:46:58 dist-test-slave-kvfs krb5kdc[19498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150018, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:46:58.675364 19514 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:46:58.675980 19514 webserver.cc:460] Webserver started at http://127.0.0.1:34753/ using document root <none> and password file <none>
I20260808 00:46:58.676311 19514 fs_manager.cc:359] Metadata directory not provided
I20260808 00:46:58.676393 19514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:46:58.676568 19514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:46:58.677878 19514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "214e841d4d654ba99024a9cfdc985f26"
format_stamp: "Formatted at 2026-08-08 00:46:58 on dist-test-slave-kvfs"
server_key: "e2c78774bf23a6f1f012de4f7d940bb7"
server_key_iv: "51c3f781ad3a47532918ad5986ac3015"
server_key_version: "encryptionkey@0"
I20260808 00:46:58.678339 19514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "214e841d4d654ba99024a9cfdc985f26"
format_stamp: "Formatted at 2026-08-08 00:46:58 on dist-test-slave-kvfs"
server_key: "e2c78774bf23a6f1f012de4f7d940bb7"
server_key_iv: "51c3f781ad3a47532918ad5986ac3015"
server_key_version: "encryptionkey@0"
I20260808 00:46:58.680159 19514 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 00:46:58.681087 19536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:46:58.681360 19514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:46:58.681494 19514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "214e841d4d654ba99024a9cfdc985f26"
format_stamp: "Formatted at 2026-08-08 00:46:58 on dist-test-slave-kvfs"
server_key: "e2c78774bf23a6f1f012de4f7d940bb7"
server_key_iv: "51c3f781ad3a47532918ad5986ac3015"
server_key_version: "encryptionkey@0"
I20260808 00:46:58.681583 19514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:46:58.686022 19514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:46:58.688529 19514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:46:58.688719 19514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:46:58.694272 19514 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39027
I20260808 00:46:58.694306 19620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39027 every 8 connection(s)
I20260808 00:46:58.694778 19514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:46:58.696161 19622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:46:58.698796 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 19514
I20260808 00:46:58.698975 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:46:58.699077 19622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26: Bootstrap starting.
I20260808 00:46:58.699236 20901 external_mini_cluster.cc:1428] Setting key c8edad5e95098cdbda38f46557be219d
I20260808 00:46:58.699946 19622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26: Neither blocks nor log segments found. Creating new log.
I20260808 00:46:58.700336 19622 log.cc:824] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26: Log is configured to *not* fsync() on all Append() calls
I20260808 00:46:58.701170 19622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26: No bootstrap required, opened a new log
I20260808 00:46:58.702322 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:58.699908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:59829 (local address 127.0.0.1:39027)
0808 00:46:58.700232 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:58.700241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:58.700272 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:46:58.700850 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:58.700859 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:58.700891 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:58.701074 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:58.701837 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:58.701842 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:58.701842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:58.701846 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:58.701890 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:58.701902 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:58.702030 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:58.702072 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 00:46:58.702124 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":131,"threads_started":1}
I20260808 00:46:58.703473 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:46:58.704439 19622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214e841d4d654ba99024a9cfdc985f26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39027 } }
I20260808 00:46:58.704639 19622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:46:58.704686 19622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 214e841d4d654ba99024a9cfdc985f26, State: Initialized, Role: FOLLOWER
I20260808 00:46:58.704861 19622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214e841d4d654ba99024a9cfdc985f26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39027 } }
I20260808 00:46:58.704999 19622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:46:58.705049 19622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:46:58.705106 19622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:46:58.706019 19622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214e841d4d654ba99024a9cfdc985f26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39027 } }
I20260808 00:46:58.706166 19622 leader_election.cc:302] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 214e841d4d654ba99024a9cfdc985f26; no voters:
I20260808 00:46:58.706368 19622 leader_election.cc:288] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:46:58.706604 19625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:46:58.705277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:44085 (local address 127.0.0.1:39027)
0808 00:46:58.705345 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:46:58.705347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:46:58.705486 (+ 139us) server_negotiation.cc:407] Connection header received
0808 00:46:58.705516 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:46:58.705517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:46:58.705522 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:46:58.705594 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:46:58.706239 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:46:58.706241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:46:58.706241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:46:58.706246 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:46:58.706267 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:46:58.706272 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:46:58.706341 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:46:58.706454 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:46:58.706471 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:46:58.706732 19622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:46:58.706487 19634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:46:58.706928 19634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [term 1 LEADER]: Becoming Leader. State: Replica: 214e841d4d654ba99024a9cfdc985f26, State: Running, Role: LEADER
I20260808 00:46:58.707206 19634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214e841d4d654ba99024a9cfdc985f26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39027 } }
I20260808 00:46:58.707715 19637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 214e841d4d654ba99024a9cfdc985f26. Latest consensus state: current_term: 1 leader_uuid: "214e841d4d654ba99024a9cfdc985f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214e841d4d654ba99024a9cfdc985f26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39027 } } }
I20260808 00:46:58.707813 19637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:58.707891 19636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "214e841d4d654ba99024a9cfdc985f26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "214e841d4d654ba99024a9cfdc985f26" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39027 } } }
I20260808 00:46:58.707983 19636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26 [sys.catalog]: This master's current role is: LEADER
I20260808 00:46:58.708107 19648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:46:58.708781 19648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:46:58.710708 19648 catalog_manager.cc:1347] Generated new cluster ID: 1631db5cb66e4d868aa582b447076f83
I20260808 00:46:58.710763 19648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:46:58.734620 19648 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:46:58.734930 19648 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:46:58.749603 19648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 214e841d4d654ba99024a9cfdc985f26: Generated new TSK 0
I20260808 00:46:58.749925 19648 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:46:58.757669 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 19514
2026-08-08T00:46:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1725 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:46:58 dist-test-slave-kvfs krb5kdc[19663](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:46:58 dist-test-slave-kvfs krb5kdc[19663](info): set up 2 sockets
Aug 08 00:46:58 dist-test-slave-kvfs krb5kdc[19663](info): commencing operation
krb5kdc: starting...
W20260808 00:47:00.052256 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.002s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:00 dist-test-slave-kvfs krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:47:00.211521 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:00.310577 19681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:00.310925 19681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:00.310973 19681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:00.312383 19681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:00.312466 19681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:00.312490 19681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:00.312511 19681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:00.312541 19681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:00.314342 19681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:00.314706 19681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:00.315007 19681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:00.321241 19693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:00.321280 19690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:00.321241 19689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:00.321704 19681 server_base.cc:1034] running on GCE node
I20260808 00:47:00.322058 19681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:00.322451 19681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:00.323679 19681 hybrid_clock.cc:648] HybridClock initialized: now 1786150020323647 us; error 42 us; skew 500 ppm
Aug 08 00:47:00 dist-test-slave-kvfs krb5kdc[19663](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150020, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:00.327190 19681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:47:00.327888 19681 webserver.cc:460] Webserver started at http://127.0.0.1:42157/ using document root <none> and password file <none>
I20260808 00:47:00.328255 19681 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:00.328337 19681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:00.328531 19681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:00.329986 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "63ca654ad0bb492fbb1ad8b9403948d6"
format_stamp: "Formatted at 2026-08-08 00:47:00 on dist-test-slave-kvfs"
server_key: "82b34815047c24466755dcbc784e8492"
server_key_iv: "0edd8bac804e729c6bf9b94652cf50d7"
server_key_version: "encryptionkey@0"
I20260808 00:47:00.330456 19681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "63ca654ad0bb492fbb1ad8b9403948d6"
format_stamp: "Formatted at 2026-08-08 00:47:00 on dist-test-slave-kvfs"
server_key: "82b34815047c24466755dcbc784e8492"
server_key_iv: "0edd8bac804e729c6bf9b94652cf50d7"
server_key_version: "encryptionkey@0"
I20260808 00:47:00.332191 19681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:47:00.333173 19709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:00.333416 19681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:00.333549 19681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "63ca654ad0bb492fbb1ad8b9403948d6"
format_stamp: "Formatted at 2026-08-08 00:47:00 on dist-test-slave-kvfs"
server_key: "82b34815047c24466755dcbc784e8492"
server_key_iv: "0edd8bac804e729c6bf9b94652cf50d7"
server_key_version: "encryptionkey@0"
I20260808 00:47:00.333640 19681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:00.343513 19681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:00.346077 19681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:00.346278 19681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:00.351897 19681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39801
I20260808 00:47:00.351969 19790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39801 every 8 connection(s)
I20260808 00:47:00.352505 19681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:00.353822 19792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:00.357264 19792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6: Bootstrap starting.
I20260808 00:47:00.358302 19792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:00.359004 19792 log.cc:824] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:00.359689 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 19681
I20260808 00:47:00.359829 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:00.360093 20901 external_mini_cluster.cc:1428] Setting key a899623f2e560e6c4d7ff6965264aeb8
I20260808 00:47:00.360291 19792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6: No bootstrap required, opened a new log
I20260808 00:47:00.362634 19792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ca654ad0bb492fbb1ad8b9403948d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260808 00:47:00.362829 19792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:00.362872 19792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63ca654ad0bb492fbb1ad8b9403948d6, State: Initialized, Role: FOLLOWER
I20260808 00:47:00.363058 19792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ca654ad0bb492fbb1ad8b9403948d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260808 00:47:00.363170 19792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:00.363207 19792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:00.363260 19792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:00.364725 19792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ca654ad0bb492fbb1ad8b9403948d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260808 00:47:00.364845 19792 leader_election.cc:302] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63ca654ad0bb492fbb1ad8b9403948d6; no voters:
I20260808 00:47:00.365084 19792 leader_election.cc:288] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:00.365135 19800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:00.365310 19800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [term 1 LEADER]: Becoming Leader. State: Replica: 63ca654ad0bb492fbb1ad8b9403948d6, State: Running, Role: LEADER
I20260808 00:47:00.365423 19800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ca654ad0bb492fbb1ad8b9403948d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260808 00:47:00.365489 19792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:00.365875 19800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63ca654ad0bb492fbb1ad8b9403948d6. Latest consensus state: current_term: 1 leader_uuid: "63ca654ad0bb492fbb1ad8b9403948d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ca654ad0bb492fbb1ad8b9403948d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } } }
I20260808 00:47:00.365970 19800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:00.366149 19802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63ca654ad0bb492fbb1ad8b9403948d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63ca654ad0bb492fbb1ad8b9403948d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } } }
I20260808 00:47:00.366209 19802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:00.366349 19807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:00.366987 19807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:00.369616 19807 catalog_manager.cc:1347] Generated new cluster ID: 6fccb161f5954be78901eb229747d6b3
I20260808 00:47:00.369694 19807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:47:00 dist-test-slave-kvfs krb5kdc[19663](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150020, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:47:00.375304 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:00.360813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:43353 (local address 127.0.0.1:39801)
0808 00:47:00.361018 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:00.361029 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:00.361060 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:47:00.362623 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:00.362637 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:00.362729 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:00.362931 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:00.363925 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:00.365259 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:00.366583 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:00.367270 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:00.367388 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:00.370988 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:00.370993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:00.370993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:00.371008 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:00.373229 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:00.373776 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:00.373779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:00.373780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:00.373928 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:00.374222 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:00.374223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:00.374224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:00.374603 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:00.374778 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:00.374951 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 00:47:00.375062 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260808 00:47:00.376623 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:00.386554 19798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:00.378410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:44553 (local address 127.0.0.1:39801)
0808 00:47:00.378457 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:00.378459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:00.378601 (+ 142us) server_negotiation.cc:407] Connection header received
0808 00:47:00.378690 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:00.378690 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:00.378701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:00.378729 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:00.379463 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:00.380296 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:00.381354 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:00.381701 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:00.381743 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:00.383477 (+ 1734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:00.383479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:00.383480 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:00.383488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:00.385161 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:00.385611 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:00.385612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:00.385613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:00.385674 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:00.385899 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:00.385900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:00.385900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:00.386029 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:00.386122 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:00.386365 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 00:47:00.386445 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:47:00.387823 19807 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:00.388439 19807 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:00.390727 19807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63ca654ad0bb492fbb1ad8b9403948d6: Generated new TSK 0
I20260808 00:47:00.391005 19807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:00.396167 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 19681
W20260808 00:47:00.412668 19660 connection.cc:535] client connection to 127.0.0.1:39801 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39801): Connection reset by peer (error 104)
2026-08-08T00:47:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1641 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:00 dist-test-slave-kvfs krb5kdc[19835](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:00 dist-test-slave-kvfs krb5kdc[19835](info): set up 2 sockets
Aug 08 00:47:00 dist-test-slave-kvfs krb5kdc[19835](info): commencing operation
krb5kdc: starting...
W20260808 00:47:01.930419 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.502s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:02 dist-test-slave-kvfs krb5kdc[19835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:47:02.188905 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:02.367354 19858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:02.367913 19858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:02.367978 19858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:02.370031 19858 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:02.370149 19858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:02.370201 19858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:02.370265 19858 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:02.370352 19858 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:02.373543 19858 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44391
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:02.374249 19858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:02.374759 19858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:02.385510 19871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:02.385493 19865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:02.385499 19864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:02.386176 19858 server_base.cc:1034] running on GCE node
I20260808 00:47:02.386658 19858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:02.387076 19858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:02.388273 19858 hybrid_clock.cc:648] HybridClock initialized: now 1786150022388238 us; error 37 us; skew 500 ppm
Aug 08 00:47:02 dist-test-slave-kvfs krb5kdc[19835](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150022, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:02.397737 19858 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:47:02.398638 19858 webserver.cc:460] Webserver started at http://127.0.0.1:45113/ using document root <none> and password file <none>
I20260808 00:47:02.399071 19858 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:02.399196 19858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:02.399470 19858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:02.401629 19858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "e0994ee6312e44988c71777a54f00468"
format_stamp: "Formatted at 2026-08-08 00:47:02 on dist-test-slave-kvfs"
server_key: "543ed3a57aba4c65b12fb0972ebfd19f"
server_key_iv: "5b52e8bd01839b1c2fb78e6382116f03"
server_key_version: "encryptionkey@0"
I20260808 00:47:02.402334 19858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "e0994ee6312e44988c71777a54f00468"
format_stamp: "Formatted at 2026-08-08 00:47:02 on dist-test-slave-kvfs"
server_key: "543ed3a57aba4c65b12fb0972ebfd19f"
server_key_iv: "5b52e8bd01839b1c2fb78e6382116f03"
server_key_version: "encryptionkey@0"
I20260808 00:47:02.405583 19858 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 00:47:02.407155 19879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:02.407541 19858 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:02.407764 19858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "e0994ee6312e44988c71777a54f00468"
format_stamp: "Formatted at 2026-08-08 00:47:02 on dist-test-slave-kvfs"
server_key: "543ed3a57aba4c65b12fb0972ebfd19f"
server_key_iv: "5b52e8bd01839b1c2fb78e6382116f03"
server_key_version: "encryptionkey@0"
I20260808 00:47:02.407929 19858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:02.435694 19858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:02.440164 19858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:02.440465 19858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:02.449534 19858 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44391
I20260808 00:47:02.449620 19968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44391 every 8 connection(s)
I20260808 00:47:02.450476 19858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:02.452579 19970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:02.453883 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 19858
I20260808 00:47:02.454167 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:02.454577 20901 external_mini_cluster.cc:1428] Setting key 7e14f98f5090664f9b059abd0495fbb5
I20260808 00:47:02.458700 19970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468: Bootstrap starting.
I20260808 00:47:02.460338 19970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:02.461141 19970 log.cc:824] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:02.462664 19970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468: No bootstrap required, opened a new log
I20260808 00:47:02.465881 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:02.455519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:60497 (local address 127.0.0.1:44391)
0808 00:47:02.455869 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:02.455897 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:02.455977 (+ 80us) server_negotiation.cc:407] Connection header received
0808 00:47:02.457060 (+ 1083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:02.457081 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:02.457180 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:02.457478 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:02.458034 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:02.459739 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:02.461956 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:02.462786 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:02.462954 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:02.464414 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:02.464431 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:02.464431 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:02.464782 (+ 351us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:02.464870 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:02.464903 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:02.465073 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:02.465431 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 00:47:02.465512 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":160,"threads_started":1}
I20260808 00:47:02.467350 19970 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0994ee6312e44988c71777a54f00468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } }
I20260808 00:47:02.467662 19970 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:02.467780 19970 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0994ee6312e44988c71777a54f00468, State: Initialized, Role: FOLLOWER
I20260808 00:47:02.468099 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:02.468077 19970 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0994ee6312e44988c71777a54f00468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } }
I20260808 00:47:02.468267 19970 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:02.468327 19970 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:02.468420 19970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:02.470225 19970 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0994ee6312e44988c71777a54f00468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } }
I20260808 00:47:02.470490 19970 leader_election.cc:302] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0994ee6312e44988c71777a54f00468; no voters:
I20260808 00:47:02.470867 19970 leader_election.cc:288] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:02.470890 19979 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:02.471186 19979 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [term 1 LEADER]: Becoming Leader. State: Replica: e0994ee6312e44988c71777a54f00468, State: Running, Role: LEADER
I20260808 00:47:02.471300 19970 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:02.471369 19979 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0994ee6312e44988c71777a54f00468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } }
I20260808 00:47:02.472196 19983 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0994ee6312e44988c71777a54f00468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0994ee6312e44988c71777a54f00468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } } }
I20260808 00:47:02.472344 19983 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:02.472587 19983 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0994ee6312e44988c71777a54f00468. Latest consensus state: current_term: 1 leader_uuid: "e0994ee6312e44988c71777a54f00468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0994ee6312e44988c71777a54f00468" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44391 } } }
I20260808 00:47:02.472667 19983 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468 [sys.catalog]: This master's current role is: LEADER
W20260808 00:47:02.474519 20000 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:47:02.474640 20000 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:47:02.474753 19991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:02.475072 19991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:02.477814 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:02.471274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:58571 (local address 127.0.0.1:44391)
0808 00:47:02.471318 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:02.471321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:02.472318 (+ 997us) server_negotiation.cc:407] Connection header received
0808 00:47:02.472660 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:02.472662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:02.472698 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:02.472789 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:02.473257 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:02.474027 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:02.475430 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:02.475772 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:02.475928 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:02.477290 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:02.477292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:02.477293 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:02.477307 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:02.477328 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:02.477343 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:02.477461 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:02.477639 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 00:47:02.477663 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:47:02.479043 19991 catalog_manager.cc:1347] Generated new cluster ID: 28ed755298fa48c881d926135cb0b384
I20260808 00:47:02.479140 19991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:02.500954 19991 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:02.502398 19991 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:02.516431 19991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0994ee6312e44988c71777a54f00468: Generated new TSK 0
I20260808 00:47:02.516955 19991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:02.523027 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 19858
W20260808 00:47:02.554765 19833 connection.cc:535] client connection to 127.0.0.1:44391 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44391): Connection reset by peer (error 104)
2026-08-08T00:47:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2144 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:02 dist-test-slave-kvfs krb5kdc[20006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:02 dist-test-slave-kvfs krb5kdc[20006](info): set up 2 sockets
Aug 08 00:47:02 dist-test-slave-kvfs krb5kdc[20006](info): commencing operation
krb5kdc: starting...
W20260808 00:47:03.851423 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.004s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:03 dist-test-slave-kvfs krb5kdc[20006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:47:04.003918 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:04.099504 20020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:04.099802 20020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:04.099843 20020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:04.101099 20020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:04.101171 20020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:04.101197 20020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:04.101218 20020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:04.101246 20020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:04.102850 20020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:04.103161 20020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:04.103439 20020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:04.108970 20033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:04.109184 20020 server_base.cc:1034] running on GCE node
W20260808 00:47:04.108984 20038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:04.109009 20032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:04.109645 20020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:04.110033 20020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:04.111225 20020 hybrid_clock.cc:648] HybridClock initialized: now 1786150024111205 us; error 40 us; skew 500 ppm
Aug 08 00:47:04 dist-test-slave-kvfs krb5kdc[20006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150024, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:04.114101 20020 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:47:04.114643 20020 webserver.cc:460] Webserver started at http://127.0.0.1:35813/ using document root <none> and password file <none>
I20260808 00:47:04.114924 20020 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:04.115013 20020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:04.115167 20020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:04.116472 20020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "2fda033aef3c4967a453dd5c91689e3a"
format_stamp: "Formatted at 2026-08-08 00:47:04 on dist-test-slave-kvfs"
server_key: "88d0a5476417ddb90ab4409458fa6f56"
server_key_iv: "25357ab76ff65cb88bc3d5e44a171336"
server_key_version: "encryptionkey@0"
I20260808 00:47:04.116863 20020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "2fda033aef3c4967a453dd5c91689e3a"
format_stamp: "Formatted at 2026-08-08 00:47:04 on dist-test-slave-kvfs"
server_key: "88d0a5476417ddb90ab4409458fa6f56"
server_key_iv: "25357ab76ff65cb88bc3d5e44a171336"
server_key_version: "encryptionkey@0"
I20260808 00:47:04.118440 20020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:47:04.119315 20049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:04.119522 20020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:04.119640 20020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "2fda033aef3c4967a453dd5c91689e3a"
format_stamp: "Formatted at 2026-08-08 00:47:04 on dist-test-slave-kvfs"
server_key: "88d0a5476417ddb90ab4409458fa6f56"
server_key_iv: "25357ab76ff65cb88bc3d5e44a171336"
server_key_version: "encryptionkey@0"
I20260808 00:47:04.119722 20020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:04.123467 20020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:04.125648 20020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:04.125818 20020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:04.130882 20020 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32781
I20260808 00:47:04.130915 20119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32781 every 8 connection(s)
I20260808 00:47:04.131342 20020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:04.132644 20120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:04.135553 20120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a: Bootstrap starting.
I20260808 00:47:04.136387 20120 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:04.136734 20120 log.cc:824] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:04.137640 20120 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a: No bootstrap required, opened a new log
I20260808 00:47:04.139544 20120 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda033aef3c4967a453dd5c91689e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32781 } }
I20260808 00:47:04.139720 20120 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:04.139746 20120 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fda033aef3c4967a453dd5c91689e3a, State: Initialized, Role: FOLLOWER
I20260808 00:47:04.139933 20120 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda033aef3c4967a453dd5c91689e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32781 } }
I20260808 00:47:04.140005 20120 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:04.140092 20120 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:04.140153 20120 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:04.140847 20120 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda033aef3c4967a453dd5c91689e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32781 } }
I20260808 00:47:04.140975 20120 leader_election.cc:302] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fda033aef3c4967a453dd5c91689e3a; no voters:
I20260808 00:47:04.141196 20120 leader_election.cc:288] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:04.141271 20128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:04.141423 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 20020
I20260808 00:47:04.141485 20128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [term 1 LEADER]: Becoming Leader. State: Replica: 2fda033aef3c4967a453dd5c91689e3a, State: Running, Role: LEADER
I20260808 00:47:04.141532 20120 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:04.141573 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:04.141621 20128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda033aef3c4967a453dd5c91689e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32781 } }
I20260808 00:47:04.141793 20901 external_mini_cluster.cc:1428] Setting key a2fa8f6d4e3df793209e6abe72d0457c
I20260808 00:47:04.142081 20128 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fda033aef3c4967a453dd5c91689e3a. Latest consensus state: current_term: 1 leader_uuid: "2fda033aef3c4967a453dd5c91689e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda033aef3c4967a453dd5c91689e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32781 } } }
I20260808 00:47:04.142189 20128 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:04.142379 20129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fda033aef3c4967a453dd5c91689e3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fda033aef3c4967a453dd5c91689e3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32781 } } }
I20260808 00:47:04.142474 20129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:04.142470 20140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:04.143265 20140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:04.145135 20151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:04.142819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:51647 (local address 127.0.0.1:32781)
0808 00:47:04.143300 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:04.143311 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:04.143380 (+ 69us) server_negotiation.cc:407] Connection header received
0808 00:47:04.143545 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:04.143553 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:04.143589 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:04.143741 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:04.144579 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:04.144584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:04.144585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:04.144590 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:04.144647 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:04.144665 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:04.144747 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:04.144855 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 00:47:04.144914 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":99,"threads_started":1}
I20260808 00:47:04.145481 20140 catalog_manager.cc:1347] Generated new cluster ID: 35a9120dbdaf4e64ab8734955258c663
I20260808 00:47:04.145536 20140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:04.146106 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:04.153537 20151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:04.148502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.146:43153 (local address 127.0.0.1:32781)
0808 00:47:04.148547 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:04.148549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:04.152285 (+ 3736us) server_negotiation.cc:407] Connection header received
0808 00:47:04.152386 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:04.152387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:04.152396 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:04.152454 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:04.153216 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:04.153218 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:04.153218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:04.153223 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:04.153243 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:04.153248 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:04.153318 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:04.153378 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 00:47:04.153398 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:47:04.165984 20140 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:04.166236 20140 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:04.177179 20140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2fda033aef3c4967a453dd5c91689e3a: Generated new TSK 0
I20260808 00:47:04.177486 20140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:04.183027 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 20020
2026-08-08T00:47:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1635 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:04 dist-test-slave-kvfs krb5kdc[20169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:04 dist-test-slave-kvfs krb5kdc[20169](info): set up 2 sockets
Aug 08 00:47:04 dist-test-slave-kvfs krb5kdc[20169](info): commencing operation
krb5kdc: starting...
W20260808 00:47:05.473388 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:05 dist-test-slave-kvfs krb5kdc[20169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:47:05.623680 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:37441
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:44821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:37441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:05.720788 20192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:05.721115 20192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:05.721160 20192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:05.722411 20192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:05.722494 20192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:05.722519 20192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:05.722543 20192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:05.722571 20192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:05.724269 20192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:44821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:37441
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:37441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:05.724627 20192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:05.724928 20192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:05.730770 20204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:05.730872 20192 server_base.cc:1034] running on GCE node
W20260808 00:47:05.730798 20201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:05.730765 20200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:05.731400 20192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:05.731760 20192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:05.732936 20192 hybrid_clock.cc:648] HybridClock initialized: now 1786150025732903 us; error 70 us; skew 500 ppm
Aug 08 00:47:05 dist-test-slave-kvfs krb5kdc[20169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150025, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:05.736274 20192 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:47:05.736810 20192 webserver.cc:460] Webserver started at http://127.20.105.126:46447/ using document root <none> and password file <none>
I20260808 00:47:05.737110 20192 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:05.737190 20192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:05.737349 20192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:05.738619 20192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "c56688ea7d7a47b18019329aa08463d1"
format_stamp: "Formatted at 2026-08-08 00:47:05 on dist-test-slave-kvfs"
server_key: "8b6da44ba658e8ba502dc11824accdb0"
server_key_iv: "6c77f5e4dddd0082ae942d60b376a691"
server_key_version: "encryptionkey@0"
I20260808 00:47:05.739040 20192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "c56688ea7d7a47b18019329aa08463d1"
format_stamp: "Formatted at 2026-08-08 00:47:05 on dist-test-slave-kvfs"
server_key: "8b6da44ba658e8ba502dc11824accdb0"
server_key_iv: "6c77f5e4dddd0082ae942d60b376a691"
server_key_version: "encryptionkey@0"
I20260808 00:47:05.740710 20192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260808 00:47:05.741818 20214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:05.742064 20192 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:47:05.742189 20192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "c56688ea7d7a47b18019329aa08463d1"
format_stamp: "Formatted at 2026-08-08 00:47:05 on dist-test-slave-kvfs"
server_key: "8b6da44ba658e8ba502dc11824accdb0"
server_key_iv: "6c77f5e4dddd0082ae942d60b376a691"
server_key_version: "encryptionkey@0"
I20260808 00:47:05.742275 20192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:05.772596 20192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:05.775345 20192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:05.775552 20192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:05.781402 20192 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:37441
I20260808 00:47:05.781464 20296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:37441 every 8 connection(s)
I20260808 00:47:05.781970 20192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:05.783098 20297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:05.786116 20297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1: Bootstrap starting.
I20260808 00:47:05.786900 20297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:05.787261 20297 log.cc:824] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:05.788245 20297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1: No bootstrap required, opened a new log
I20260808 00:47:05.790176 20297 raft_consensus.cc:348] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56688ea7d7a47b18019329aa08463d1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 37441 } }
I20260808 00:47:05.790352 20297 raft_consensus.cc:374] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:05.790378 20297 raft_consensus.cc:729] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56688ea7d7a47b18019329aa08463d1, State: Initialized, Role: FOLLOWER
I20260808 00:47:05.790598 20297 consensus_queue.cc:260] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56688ea7d7a47b18019329aa08463d1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 37441 } }
I20260808 00:47:05.790691 20297 raft_consensus.cc:388] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:05.790742 20297 raft_consensus.cc:482] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:05.790797 20297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:05.791886 20297 raft_consensus.cc:504] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56688ea7d7a47b18019329aa08463d1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 37441 } }
I20260808 00:47:05.791992 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 20192
I20260808 00:47:05.792052 20297 leader_election.cc:302] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c56688ea7d7a47b18019329aa08463d1; no voters:
I20260808 00:47:05.792171 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:05.792276 20297 leader_election.cc:288] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:05.792331 20302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:05.792443 20901 external_mini_cluster.cc:1428] Setting key a1478e618c72c2907a07eb320e86e79a
I20260808 00:47:05.792552 20302 raft_consensus.cc:686] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [term 1 LEADER]: Becoming Leader. State: Replica: c56688ea7d7a47b18019329aa08463d1, State: Running, Role: LEADER
I20260808 00:47:05.792629 20297 sys_catalog.cc:564] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:05.792666 20302 consensus_queue.cc:237] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56688ea7d7a47b18019329aa08463d1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 37441 } }
I20260808 00:47:05.793166 20303 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c56688ea7d7a47b18019329aa08463d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56688ea7d7a47b18019329aa08463d1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 37441 } } }
I20260808 00:47:05.793260 20303 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:05.793517 20311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:05.794289 20311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:05.793509 20304 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c56688ea7d7a47b18019329aa08463d1. Latest consensus state: current_term: 1 leader_uuid: "c56688ea7d7a47b18019329aa08463d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56688ea7d7a47b18019329aa08463d1" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 37441 } } }
I20260808 00:47:05.795394 20304 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:05.797138 20311 catalog_manager.cc:1347] Generated new cluster ID: 9984c2c9ceb74c0a818ce1593bba3060
I20260808 00:47:05.797199 20311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:47:05 dist-test-slave-kvfs krb5kdc[20169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:47:05.804970 20311 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:05.805760 20311 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:05.806231 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:05.793116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51575 (local address 127.20.105.126:37441)
0808 00:47:05.793684 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:05.793696 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:05.793739 (+ 43us) server_negotiation.cc:407] Connection header received
0808 00:47:05.793934 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:05.793944 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:05.794030 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:05.795224 (+ 1194us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:05.795352 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:05.796656 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:05.797832 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:05.798484 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:05.798591 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:05.802006 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:05.802014 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:05.802015 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:05.802034 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:05.804351 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:05.804842 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:05.804846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:05.804846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:05.805001 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:05.805265 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:05.805266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:05.805267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:05.805529 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:05.805754 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:05.805853 (+ 99us) server_negotiation.cc:299] Negotiation successful
0808 00:47:05.805995 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":83,"threads_started":1}
I20260808 00:47:05.807294 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:05.815716 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:05.809116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32871 (local address 127.20.105.126:37441)
0808 00:47:05.809158 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:05.809160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:05.809284 (+ 124us) server_negotiation.cc:407] Connection header received
0808 00:47:05.809361 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:05.809362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:05.809372 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:05.809412 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:05.809968 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:05.810536 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:05.811293 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:05.811539 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:05.811623 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:05.812958 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:05.812960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:05.812961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:05.812968 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:05.814460 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:05.814845 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:05.814846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:05.814847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:05.814911 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:05.815140 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:05.815141 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:05.815141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:05.815262 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:05.815383 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:05.815512 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 00:47:05.815583 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:47:05.818284 20311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c56688ea7d7a47b18019329aa08463d1: Generated new TSK 0
I20260808 00:47:05.818514 20311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:05.824831 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 20192
W20260808 00:47:05.842613 20166 connection.cc:535] client connection to 127.20.105.126:37441 recv error: Network error: failed to read from TLS socket (remote: 127.20.105.126:37441): Connection reset by peer (error 104)
2026-08-08T00:47:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:05 dist-test-slave-kvfs krb5kdc[20337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:05 dist-test-slave-kvfs krb5kdc[20337](info): set up 2 sockets
Aug 08 00:47:05 dist-test-slave-kvfs krb5kdc[20337](info): commencing operation
krb5kdc: starting...
W20260808 00:47:07.147910 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:07 dist-test-slave-kvfs krb5kdc[20337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:47:07.300169 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:38419
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:37935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:38419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:07.398295 20356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:07.398617 20356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:07.398658 20356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:07.399996 20356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:07.400103 20356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:07.400130 20356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:07.400151 20356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:07.400172 20356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:07.401818 20356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:37935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:38419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:38419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20356
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:07.402171 20356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:07.402456 20356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:47:07.408500 20356 server_base.cc:1034] running on GCE node
W20260808 00:47:07.408439 20368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:07.408432 20363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:07.408437 20364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:07.409149 20356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:07.409505 20356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:07.410681 20356 hybrid_clock.cc:648] HybridClock initialized: now 1786150027410653 us; error 58 us; skew 500 ppm
Aug 08 00:47:07 dist-test-slave-kvfs krb5kdc[20337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150027, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:07.419646 20356 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:47:07.420290 20356 webserver.cc:460] Webserver started at http://127.20.105.126:44501/ using document root <none> and password file <none>
I20260808 00:47:07.420558 20356 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:07.420632 20356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:07.420794 20356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:07.422101 20356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "7c17e2b4b9b74c42b5d551c650cd6455"
format_stamp: "Formatted at 2026-08-08 00:47:07 on dist-test-slave-kvfs"
server_key: "9ff74823a72d0bf5349b75e847a36e88"
server_key_iv: "40db947376b1398c8eca116e04a7b4b1"
server_key_version: "encryptionkey@0"
I20260808 00:47:07.422520 20356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "7c17e2b4b9b74c42b5d551c650cd6455"
format_stamp: "Formatted at 2026-08-08 00:47:07 on dist-test-slave-kvfs"
server_key: "9ff74823a72d0bf5349b75e847a36e88"
server_key_iv: "40db947376b1398c8eca116e04a7b4b1"
server_key_version: "encryptionkey@0"
I20260808 00:47:07.424137 20356 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260808 00:47:07.424958 20385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:07.425169 20356 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:07.425282 20356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "7c17e2b4b9b74c42b5d551c650cd6455"
format_stamp: "Formatted at 2026-08-08 00:47:07 on dist-test-slave-kvfs"
server_key: "9ff74823a72d0bf5349b75e847a36e88"
server_key_iv: "40db947376b1398c8eca116e04a7b4b1"
server_key_version: "encryptionkey@0"
I20260808 00:47:07.425361 20356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:07.440761 20356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:07.443332 20356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:07.443527 20356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:07.448741 20356 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:38419
I20260808 00:47:07.448783 20462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:38419 every 8 connection(s)
I20260808 00:47:07.449247 20356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:07.450390 20463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:07.453301 20463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455: Bootstrap starting.
I20260808 00:47:07.453989 20463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:07.454273 20463 log.cc:824] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:07.455125 20463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455: No bootstrap required, opened a new log
I20260808 00:47:07.457376 20463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c17e2b4b9b74c42b5d551c650cd6455" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38419 } }
I20260808 00:47:07.457559 20463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:07.457587 20463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c17e2b4b9b74c42b5d551c650cd6455, State: Initialized, Role: FOLLOWER
I20260808 00:47:07.457856 20463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c17e2b4b9b74c42b5d551c650cd6455" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38419 } }
I20260808 00:47:07.457959 20463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:07.457985 20463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:07.458055 20463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:07.458215 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 20356
I20260808 00:47:07.458369 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:07.458580 20901 external_mini_cluster.cc:1428] Setting key b5dd62098d0721df1eb15fc26d8944a2
I20260808 00:47:07.459519 20463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c17e2b4b9b74c42b5d551c650cd6455" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38419 } }
I20260808 00:47:07.459656 20463 leader_election.cc:302] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c17e2b4b9b74c42b5d551c650cd6455; no voters:
I20260808 00:47:07.459836 20463 leader_election.cc:288] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:07.459884 20473 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:07.460124 20473 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [term 1 LEADER]: Becoming Leader. State: Replica: 7c17e2b4b9b74c42b5d551c650cd6455, State: Running, Role: LEADER
I20260808 00:47:07.460193 20463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:07.460285 20473 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c17e2b4b9b74c42b5d551c650cd6455" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38419 } }
I20260808 00:47:07.460673 20473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c17e2b4b9b74c42b5d551c650cd6455. Latest consensus state: current_term: 1 leader_uuid: "7c17e2b4b9b74c42b5d551c650cd6455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c17e2b4b9b74c42b5d551c650cd6455" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38419 } } }
I20260808 00:47:07.460695 20476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c17e2b4b9b74c42b5d551c650cd6455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c17e2b4b9b74c42b5d551c650cd6455" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 38419 } } }
I20260808 00:47:07.460772 20473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:07.460776 20476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:07.461047 20484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:07.462342 20484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:07.464306 20484 catalog_manager.cc:1347] Generated new cluster ID: acfb3ae5b3bd4e97ac6295a139efd809
I20260808 00:47:07.464363 20484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:07.465878 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:07.459246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53377 (local address 127.20.105.126:38419)
0808 00:47:07.459505 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:07.459519 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:07.459563 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:47:07.459715 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:07.459723 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:07.459812 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:07.459993 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:07.460355 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:07.462183 (+ 1828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:07.463521 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:07.464168 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:07.464283 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:07.465116 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:07.465120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:07.465120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:07.465351 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:07.465391 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:07.465404 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:07.465500 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:07.465592 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 00:47:07.465626 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":109,"threads_started":1}
I20260808 00:47:07.466921 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:07.472239 20470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:07.468651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59401 (local address 127.20.105.126:38419)
0808 00:47:07.468714 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:07.468716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:07.468847 (+ 131us) server_negotiation.cc:407] Connection header received
0808 00:47:07.468914 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:07.468915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:07.468928 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:07.468985 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:07.469157 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:07.469987 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:07.470765 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:07.471022 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:07.471106 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:07.471878 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:07.471879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:07.471880 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:07.471885 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:07.471903 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:07.471907 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:07.471957 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:07.472101 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 00:47:07.472119 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:47:07.473819 20484 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:07.474346 20484 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:07.481279 20484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c17e2b4b9b74c42b5d551c650cd6455: Generated new TSK 0
I20260808 00:47:07.481503 20484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:07.487951 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 20356
W20260808 00:47:07.502266 20335 connection.cc:535] client connection to 127.20.105.126:38419 recv error: Network error: failed to read from TLS socket (remote: 127.20.105.126:38419): Connection reset by peer (error 104)
2026-08-08T00:47:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1659 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:07 dist-test-slave-kvfs krb5kdc[20507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:07 dist-test-slave-kvfs krb5kdc[20507](info): set up 2 sockets
Aug 08 00:47:07 dist-test-slave-kvfs krb5kdc[20507](info): commencing operation
krb5kdc: starting...
W20260808 00:47:08.767684 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:08 dist-test-slave-kvfs krb5kdc[20507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150028, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:47:08.928746 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:35017
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:35815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:35017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:09.025511 20527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:09.025830 20527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:09.025871 20527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:09.027109 20527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:09.027187 20527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:09.027211 20527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:09.027232 20527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:09.027281 20527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:09.028954 20527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:35815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:35017
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:35017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:09.029294 20527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:09.029568 20527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:09.035432 20537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:09.035444 20539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:09.035622 20535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:09.035943 20527 server_base.cc:1034] running on GCE node
I20260808 00:47:09.036327 20527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:09.036705 20527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:09.037887 20527 hybrid_clock.cc:648] HybridClock initialized: now 1786150029037878 us; error 41 us; skew 500 ppm
Aug 08 00:47:09 dist-test-slave-kvfs krb5kdc[20507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150029, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:09.041145 20527 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:47:09.041730 20527 webserver.cc:460] Webserver started at http://127.20.105.126:42297/ using document root <none> and password file <none>
I20260808 00:47:09.041980 20527 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:09.042028 20527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:09.042148 20527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:09.043507 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "b6d17d7cecd14a98a8a4f9cc417371f5"
format_stamp: "Formatted at 2026-08-08 00:47:09 on dist-test-slave-kvfs"
server_key: "b889fd0e6bbe5a424d017dd99e7db666"
server_key_iv: "0abb8fcdf9372ab655db87fee57972d4"
server_key_version: "encryptionkey@0"
I20260808 00:47:09.043931 20527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "b6d17d7cecd14a98a8a4f9cc417371f5"
format_stamp: "Formatted at 2026-08-08 00:47:09 on dist-test-slave-kvfs"
server_key: "b889fd0e6bbe5a424d017dd99e7db666"
server_key_iv: "0abb8fcdf9372ab655db87fee57972d4"
server_key_version: "encryptionkey@0"
I20260808 00:47:09.045735 20527 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260808 00:47:09.046747 20553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:09.047019 20527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:47:09.047125 20527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "b6d17d7cecd14a98a8a4f9cc417371f5"
format_stamp: "Formatted at 2026-08-08 00:47:09 on dist-test-slave-kvfs"
server_key: "b889fd0e6bbe5a424d017dd99e7db666"
server_key_iv: "0abb8fcdf9372ab655db87fee57972d4"
server_key_version: "encryptionkey@0"
I20260808 00:47:09.047204 20527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:09.051535 20527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:09.054386 20527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:09.054529 20527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:09.060086 20527 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:35017
I20260808 00:47:09.060184 20625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:35017 every 8 connection(s)
I20260808 00:47:09.060585 20527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:09.061867 20627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:09.064924 20627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5: Bootstrap starting.
I20260808 00:47:09.065689 20627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:09.066030 20627 log.cc:824] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:09.066512 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 20527
I20260808 00:47:09.066685 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:09.066962 20901 external_mini_cluster.cc:1428] Setting key 92a3d72441947068672b57f3b4579c4c
I20260808 00:47:09.066975 20627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5: No bootstrap required, opened a new log
I20260808 00:47:09.069801 20627 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d17d7cecd14a98a8a4f9cc417371f5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35017 } }
I20260808 00:47:09.070043 20627 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:09.070111 20627 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d17d7cecd14a98a8a4f9cc417371f5, State: Initialized, Role: FOLLOWER
I20260808 00:47:09.070290 20627 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d17d7cecd14a98a8a4f9cc417371f5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35017 } }
I20260808 00:47:09.070415 20627 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:09.070464 20627 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:09.070478 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:09.067678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41741 (local address 127.20.105.126:35017)
0808 00:47:09.067903 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:09.067912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:09.067943 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:47:09.068624 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:09.068632 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:09.068675 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:09.068819 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:09.069631 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:09.069636 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:09.069636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:09.069641 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:09.069688 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:09.069700 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:09.069768 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:09.069891 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 00:47:09.069967 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":127,"threads_started":1}
I20260808 00:47:09.070520 20627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:09.071453 20627 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d17d7cecd14a98a8a4f9cc417371f5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35017 } }
I20260808 00:47:09.071592 20627 leader_election.cc:302] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d17d7cecd14a98a8a4f9cc417371f5; no voters:
I20260808 00:47:09.071806 20627 leader_election.cc:288] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:09.071808 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:09.072111 20633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:09.072160 20627 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:09.072289 20633 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [term 1 LEADER]: Becoming Leader. State: Replica: b6d17d7cecd14a98a8a4f9cc417371f5, State: Running, Role: LEADER
I20260808 00:47:09.072436 20633 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d17d7cecd14a98a8a4f9cc417371f5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35017 } }
I20260808 00:47:09.072918 20633 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d17d7cecd14a98a8a4f9cc417371f5. Latest consensus state: current_term: 1 leader_uuid: "b6d17d7cecd14a98a8a4f9cc417371f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d17d7cecd14a98a8a4f9cc417371f5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35017 } } }
I20260808 00:47:09.073035 20635 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6d17d7cecd14a98a8a4f9cc417371f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d17d7cecd14a98a8a4f9cc417371f5" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35017 } } }
I20260808 00:47:09.073052 20633 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:09.073120 20635 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:09.073350 20650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:09.074011 20650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:09.075730 20631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:09.074409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59337 (local address 127.20.105.126:35017)
0808 00:47:09.074452 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:09.074454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:09.074624 (+ 170us) server_negotiation.cc:407] Connection header received
0808 00:47:09.074701 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:09.074702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:09.074709 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:09.074768 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:09.075502 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:09.075504 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:09.075504 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:09.075509 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:09.075526 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:09.075531 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:09.075590 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:09.075641 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 00:47:09.075657 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:47:09.076308 20650 catalog_manager.cc:1347] Generated new cluster ID: 303861a93bef422f9975247d5d9f4f8b
I20260808 00:47:09.076360 20650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:09.093443 20650 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:09.093717 20650 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:09.097675 20650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6d17d7cecd14a98a8a4f9cc417371f5: Generated new TSK 0
I20260808 00:47:09.097937 20650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:09.105967 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 20527
2026-08-08T00:47:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1624 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:09 dist-test-slave-kvfs krb5kdc[20676](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:09 dist-test-slave-kvfs krb5kdc[20676](info): set up 2 sockets
Aug 08 00:47:09 dist-test-slave-kvfs krb5kdc[20676](info): commencing operation
krb5kdc: starting...
W20260808 00:47:10.417403 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:10 dist-test-slave-kvfs krb5kdc[20676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:47:10.571849 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:36305
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:36305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:10.670833 20692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:10.671176 20692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:10.671224 20692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:10.672626 20692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:10.672712 20692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:10.672737 20692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:10.672760 20692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:10.672816 20692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:10.674619 20692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:36305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:36305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:10.674985 20692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:10.675292 20692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:10.681500 20701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:10.681452 20700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:10.681453 20706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:10.681852 20692 server_base.cc:1034] running on GCE node
I20260808 00:47:10.682224 20692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:10.682626 20692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:10.683869 20692 hybrid_clock.cc:648] HybridClock initialized: now 1786150030683844 us; error 59 us; skew 500 ppm
Aug 08 00:47:10 dist-test-slave-kvfs krb5kdc[20676](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150030, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:10.687150 20692 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:47:10.687711 20692 webserver.cc:460] Webserver started at http://127.20.105.126:40311/ using document root <none> and password file <none>
I20260808 00:47:10.688000 20692 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:10.688130 20692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:10.688316 20692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:10.689750 20692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "8a9a9613caf545329b7fce2ccaeee709"
format_stamp: "Formatted at 2026-08-08 00:47:10 on dist-test-slave-kvfs"
server_key: "ea681f46c5841264395fadf3302d2b0c"
server_key_iv: "48a5e7a909b0224c7a96f31870cd5f42"
server_key_version: "encryptionkey@0"
I20260808 00:47:10.690203 20692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "8a9a9613caf545329b7fce2ccaeee709"
format_stamp: "Formatted at 2026-08-08 00:47:10 on dist-test-slave-kvfs"
server_key: "ea681f46c5841264395fadf3302d2b0c"
server_key_iv: "48a5e7a909b0224c7a96f31870cd5f42"
server_key_version: "encryptionkey@0"
I20260808 00:47:10.691880 20692 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260808 00:47:10.692909 20718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:10.693184 20692 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:47:10.693320 20692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "8a9a9613caf545329b7fce2ccaeee709"
format_stamp: "Formatted at 2026-08-08 00:47:10 on dist-test-slave-kvfs"
server_key: "ea681f46c5841264395fadf3302d2b0c"
server_key_iv: "48a5e7a909b0224c7a96f31870cd5f42"
server_key_version: "encryptionkey@0"
I20260808 00:47:10.693418 20692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:10.705958 20692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:10.709088 20692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:10.709303 20692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:10.714819 20692 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:36305
I20260808 00:47:10.714890 20793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:36305 every 8 connection(s)
I20260808 00:47:10.715373 20692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:10.716648 20794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:10.719635 20794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709: Bootstrap starting.
I20260808 00:47:10.720326 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 20692
I20260808 00:47:10.720366 20794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:10.720475 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:10.720675 20901 external_mini_cluster.cc:1428] Setting key c042356cefae384e137587d91a070126
I20260808 00:47:10.720808 20794 log.cc:824] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:10.721804 20794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709: No bootstrap required, opened a new log
I20260808 00:47:10.724126 20794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9a9613caf545329b7fce2ccaeee709" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36305 } }
I20260808 00:47:10.724298 20794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:10.724327 20794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a9a9613caf545329b7fce2ccaeee709, State: Initialized, Role: FOLLOWER
I20260808 00:47:10.724507 20794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9a9613caf545329b7fce2ccaeee709" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36305 } }
I20260808 00:47:10.724611 20794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:10.724649 20794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:10.724701 20794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:10.725529 20794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9a9613caf545329b7fce2ccaeee709" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36305 } }
I20260808 00:47:10.725667 20794 leader_election.cc:302] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a9a9613caf545329b7fce2ccaeee709; no voters:
I20260808 00:47:10.725890 20794 leader_election.cc:288] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:10.725941 20804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:10.726146 20804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [term 1 LEADER]: Becoming Leader. State: Replica: 8a9a9613caf545329b7fce2ccaeee709, State: Running, Role: LEADER
I20260808 00:47:10.726219 20794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:10.726328 20804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9a9613caf545329b7fce2ccaeee709" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36305 } }
I20260808 00:47:10.727025 20805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a9a9613caf545329b7fce2ccaeee709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9a9613caf545329b7fce2ccaeee709" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36305 } } }
I20260808 00:47:10.727142 20805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:10.727685 20806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a9a9613caf545329b7fce2ccaeee709. Latest consensus state: current_term: 1 leader_uuid: "8a9a9613caf545329b7fce2ccaeee709" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a9a9613caf545329b7fce2ccaeee709" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 36305 } } }
I20260808 00:47:10.727851 20806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709 [sys.catalog]: This master's current role is: LEADER
W20260808 00:47:10.728580 20826 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:47:10.728654 20826 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:47:10.728749 20818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:10.728979 20818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:47:10 dist-test-slave-kvfs krb5kdc[20676](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:47:10.731004 20818 catalog_manager.cc:1347] Generated new cluster ID: d757947543ab4e99a4107a6d37837220
I20260808 00:47:10.731069 20818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:10.734297 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:10.721345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36645 (local address 127.20.105.126:36305)
0808 00:47:10.721536 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:10.721548 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:10.721595 (+ 47us) server_negotiation.cc:407] Connection header received
0808 00:47:10.722202 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:10.722211 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:10.722273 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:10.722446 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:10.723178 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:10.724267 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:10.725428 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:10.726051 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:10.726592 (+ 541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:10.729793 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:10.729798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:10.729798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:10.729813 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:10.732222 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:10.732734 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:10.732738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:10.732738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:10.732860 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:10.733095 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:10.733096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:10.733096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:10.733462 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:10.733705 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:10.733867 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 00:47:10.734026 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260808 00:47:10.735546 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:10.749233 20801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:10.737451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49477 (local address 127.20.105.126:36305)
0808 00:47:10.737494 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:10.737496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:10.740833 (+ 3337us) server_negotiation.cc:407] Connection header received
0808 00:47:10.740920 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:10.740920 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:10.740935 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:10.741002 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:10.741810 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:10.742654 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:10.743778 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:10.744188 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:10.744255 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:10.745976 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:10.745978 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:10.745979 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:10.745988 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:10.747745 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:10.748204 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:10.748205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:10.748205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:10.748276 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:10.748502 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:10.748502 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:10.748503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:10.748649 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:10.748754 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:10.748901 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 00:47:10.748998 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:47:10.754364 20818 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:10.755064 20818 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:10.761487 20818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a9a9613caf545329b7fce2ccaeee709: Generated new TSK 0
I20260808 00:47:10.761775 20818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:10.768311 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 20692
2026-08-08T00:47:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1662 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:10 dist-test-slave-kvfs krb5kdc[20845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:10 dist-test-slave-kvfs krb5kdc[20845](info): set up 2 sockets
Aug 08 00:47:10 dist-test-slave-kvfs krb5kdc[20845](info): commencing operation
krb5kdc: starting...
W20260808 00:47:12.094436 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:12 dist-test-slave-kvfs krb5kdc[20845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:47:12.252790 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:43805
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:46285
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:43805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:12.352368 20866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:12.352651 20866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:12.352677 20866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:12.353876 20866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:12.353916 20866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:12.353930 20866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:12.353943 20866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:12.353955 20866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:12.355530 20866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:46285
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:43805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:43805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:12.355803 20866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:12.356099 20866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:12.362094 20876 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:12.362244 20866 server_base.cc:1034] running on GCE node
W20260808 00:47:12.362061 20878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:12.362123 20874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:12.362730 20866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:12.363112 20866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:12.364287 20866 hybrid_clock.cc:648] HybridClock initialized: now 1786150032364277 us; error 49 us; skew 500 ppm
Aug 08 00:47:12 dist-test-slave-kvfs krb5kdc[20845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150032, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:12.367322 20866 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:47:12.367928 20866 webserver.cc:460] Webserver started at http://127.20.105.126:45673/ using document root <none> and password file <none>
I20260808 00:47:12.368222 20866 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:12.368301 20866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:12.368475 20866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:12.369789 20866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "cbe519c87632422c824901095eeac1fd"
format_stamp: "Formatted at 2026-08-08 00:47:12 on dist-test-slave-kvfs"
server_key: "9027bb57061d3d6866ca4aac5549dd5f"
server_key_iv: "6d41fba50840d8dca80391c2355749f3"
server_key_version: "encryptionkey@0"
I20260808 00:47:12.370240 20866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "cbe519c87632422c824901095eeac1fd"
format_stamp: "Formatted at 2026-08-08 00:47:12 on dist-test-slave-kvfs"
server_key: "9027bb57061d3d6866ca4aac5549dd5f"
server_key_iv: "6d41fba50840d8dca80391c2355749f3"
server_key_version: "encryptionkey@0"
I20260808 00:47:12.371860 20866 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:47:12.372898 20889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:12.373131 20866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:47:12.373258 20866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "cbe519c87632422c824901095eeac1fd"
format_stamp: "Formatted at 2026-08-08 00:47:12 on dist-test-slave-kvfs"
server_key: "9027bb57061d3d6866ca4aac5549dd5f"
server_key_iv: "6d41fba50840d8dca80391c2355749f3"
server_key_version: "encryptionkey@0"
I20260808 00:47:12.373342 20866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:12.397768 20866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:12.400971 20866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:12.401223 20866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:12.406895 20866 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:43805
I20260808 00:47:12.406957 20977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:43805 every 8 connection(s)
I20260808 00:47:12.407433 20866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:12.408598 20978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:12.411260 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 20866
I20260808 00:47:12.411425 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:12.411643 20901 external_mini_cluster.cc:1428] Setting key ba0d917d2c3717424ce060867f63f775
I20260808 00:47:12.411819 20978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd: Bootstrap starting.
I20260808 00:47:12.412873 20978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:12.413278 20978 log.cc:824] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:12.414292 20978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd: No bootstrap required, opened a new log
I20260808 00:47:12.416968 20978 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe519c87632422c824901095eeac1fd" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43805 } }
I20260808 00:47:12.417182 20978 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:12.417232 20978 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe519c87632422c824901095eeac1fd, State: Initialized, Role: FOLLOWER
I20260808 00:47:12.417428 20978 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe519c87632422c824901095eeac1fd" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43805 } }
I20260808 00:47:12.417515 20978 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:12.417541 20978 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:12.417618 20978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:12.418308 20978 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe519c87632422c824901095eeac1fd" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43805 } }
I20260808 00:47:12.418447 20978 leader_election.cc:302] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbe519c87632422c824901095eeac1fd; no voters:
I20260808 00:47:12.418642 20978 leader_election.cc:288] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:12.418695 20985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:12.418943 20985 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [term 1 LEADER]: Becoming Leader. State: Replica: cbe519c87632422c824901095eeac1fd, State: Running, Role: LEADER
I20260808 00:47:12.419021 20978 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:12.419116 20985 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe519c87632422c824901095eeac1fd" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43805 } }
I20260808 00:47:12.419552 20986 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbe519c87632422c824901095eeac1fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe519c87632422c824901095eeac1fd" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43805 } } }
I20260808 00:47:12.419581 20988 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbe519c87632422c824901095eeac1fd. Latest consensus state: current_term: 1 leader_uuid: "cbe519c87632422c824901095eeac1fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbe519c87632422c824901095eeac1fd" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43805 } } }
I20260808 00:47:12.419731 20988 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:12.419987 20986 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:12.420001 20998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:12.420753 20998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:12.422719 20998 catalog_manager.cc:1347] Generated new cluster ID: 03dec8d7e0f24358a1fdef7fc42685b4
I20260808 00:47:12.422803 20998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:12.424261 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:12.412334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45199 (local address 127.20.105.126:43805)
0808 00:47:12.412588 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:12.412600 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:12.412642 (+ 42us) server_negotiation.cc:407] Connection header received
0808 00:47:12.413401 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:12.413412 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:12.413496 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:12.413660 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:12.413966 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:12.415502 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:12.416459 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:12.417151 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:12.417268 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:12.423356 (+ 6088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:12.423361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:12.423362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:12.423622 (+ 260us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:12.423672 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:12.423690 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:12.423774 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:12.423857 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 00:47:12.423941 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":119,"threads_started":1}
I20260808 00:47:12.425396 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:12.431473 20981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:12.427371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59593 (local address 127.20.105.126:43805)
0808 00:47:12.427415 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:12.427417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:12.427593 (+ 176us) server_negotiation.cc:407] Connection header received
0808 00:47:12.427664 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:12.427665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:12.427678 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:12.427729 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:12.427915 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:12.428766 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:12.429949 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:12.430273 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:12.430343 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:12.431180 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:12.431182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:12.431182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:12.431187 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:12.431206 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:12.431210 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:12.431266 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:12.431336 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 00:47:12.431368 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:47:12.440759 20998 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:12.441632 20998 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:12.445544 20998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbe519c87632422c824901095eeac1fd: Generated new TSK 0
I20260808 00:47:12.445806 20998 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:12.451462 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 20866
2026-08-08T00:47:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1682 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:12 dist-test-slave-kvfs krb5kdc[21022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:12 dist-test-slave-kvfs krb5kdc[21022](info): set up 2 sockets
Aug 08 00:47:12 dist-test-slave-kvfs krb5kdc[21022](info): commencing operation
krb5kdc: starting...
W20260808 00:47:13.757548 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:13 dist-test-slave-kvfs krb5kdc[21022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150033, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:47:13.911635 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40301
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:41967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:40301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:14.011626 21044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:14.011902 21044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:14.011926 21044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:14.013279 21044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:14.013337 21044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:14.013352 21044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:14.013396 21044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:14.013427 21044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:14.015175 21044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:41967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:40301
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:40301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:14.015507 21044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:14.015815 21044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:14.021559 21052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:14.021574 21054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:14.021580 21058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:14.021755 21044 server_base.cc:1034] running on GCE node
I20260808 00:47:14.022140 21044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:14.022496 21044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:14.023675 21044 hybrid_clock.cc:648] HybridClock initialized: now 1786150034023663 us; error 50 us; skew 500 ppm
Aug 08 00:47:14 dist-test-slave-kvfs krb5kdc[21022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150034, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:14.026631 21044 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:47:14.027187 21044 webserver.cc:460] Webserver started at http://127.20.105.126:44189/ using document root <none> and password file <none>
I20260808 00:47:14.027473 21044 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:14.027552 21044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:14.027731 21044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:14.029135 21044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "fd8468984fae48bab0a1c9eeb349a2e0"
format_stamp: "Formatted at 2026-08-08 00:47:14 on dist-test-slave-kvfs"
server_key: "80d6050409d445cd441b0159f499dc1a"
server_key_iv: "ff0dae32e9c25b5391adef7462543a52"
server_key_version: "encryptionkey@0"
I20260808 00:47:14.029578 21044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "fd8468984fae48bab0a1c9eeb349a2e0"
format_stamp: "Formatted at 2026-08-08 00:47:14 on dist-test-slave-kvfs"
server_key: "80d6050409d445cd441b0159f499dc1a"
server_key_iv: "ff0dae32e9c25b5391adef7462543a52"
server_key_version: "encryptionkey@0"
I20260808 00:47:14.031299 21044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:47:14.032409 21068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:14.032694 21044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:14.032829 21044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "fd8468984fae48bab0a1c9eeb349a2e0"
format_stamp: "Formatted at 2026-08-08 00:47:14 on dist-test-slave-kvfs"
server_key: "80d6050409d445cd441b0159f499dc1a"
server_key_iv: "ff0dae32e9c25b5391adef7462543a52"
server_key_version: "encryptionkey@0"
I20260808 00:47:14.032927 21044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:14.037145 21044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:14.039680 21044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:14.039870 21044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:14.045156 21044 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:40301
I20260808 00:47:14.045194 21142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:40301 every 8 connection(s)
I20260808 00:47:14.045624 21044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:14.046921 21144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:14.049732 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21044
I20260808 00:47:14.049885 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:14.050107 20901 external_mini_cluster.cc:1428] Setting key aafc2f2e23fe6fe76e312b73deb3f630
I20260808 00:47:14.050100 21144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0: Bootstrap starting.
I20260808 00:47:14.050992 21144 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:14.051395 21144 log.cc:824] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:14.052428 21144 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0: No bootstrap required, opened a new log
I20260808 00:47:14.053622 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:14.050819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34967 (local address 127.20.105.126:40301)
0808 00:47:14.051137 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:14.051150 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:14.051193 (+ 43us) server_negotiation.cc:407] Connection header received
0808 00:47:14.051945 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:14.051959 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:14.052006 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:14.052226 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:14.053111 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:14.053115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:14.053115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:14.053119 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:14.053166 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:14.053177 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:14.053286 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:14.053369 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 00:47:14.053419 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":154,"threads_started":1}
I20260808 00:47:14.054785 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:14.055131 21144 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8468984fae48bab0a1c9eeb349a2e0" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40301 } }
I20260808 00:47:14.055320 21144 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:47:14.055390 21144 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd8468984fae48bab0a1c9eeb349a2e0, State: Initialized, Role: FOLLOWER
I20260808 00:47:14.055559 21144 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8468984fae48bab0a1c9eeb349a2e0" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40301 } }
I20260808 00:47:14.055662 21144 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:47:14.055711 21144 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:47:14.055763 21144 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:14.056757 21144 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8468984fae48bab0a1c9eeb349a2e0" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40301 } }
I20260808 00:47:14.056891 21144 leader_election.cc:302] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd8468984fae48bab0a1c9eeb349a2e0; no voters:
I20260808 00:47:14.057104 21144 leader_election.cc:288] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:47:14.057132 21162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:14.057308 21162 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [term 1 LEADER]: Becoming Leader. State: Replica: fd8468984fae48bab0a1c9eeb349a2e0, State: Running, Role: LEADER
I20260808 00:47:14.057420 21162 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8468984fae48bab0a1c9eeb349a2e0" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40301 } }
I20260808 00:47:14.057462 21144 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:14.057873 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:14.056599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35769 (local address 127.20.105.126:40301)
0808 00:47:14.056636 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:14.056638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:14.056793 (+ 155us) server_negotiation.cc:407] Connection header received
0808 00:47:14.056873 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:14.056874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:14.056880 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:14.056937 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:14.057602 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:14.057603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:14.057603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:47:14.057607 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:14.057622 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:47:14.057626 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:14.057691 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:14.057773 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 00:47:14.057790 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:47:14.058856 21163 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd8468984fae48bab0a1c9eeb349a2e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8468984fae48bab0a1c9eeb349a2e0" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40301 } } }
I20260808 00:47:14.058892 21165 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd8468984fae48bab0a1c9eeb349a2e0. Latest consensus state: current_term: 1 leader_uuid: "fd8468984fae48bab0a1c9eeb349a2e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd8468984fae48bab0a1c9eeb349a2e0" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 40301 } } }
I20260808 00:47:14.058965 21163 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:14.059002 21165 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0 [sys.catalog]: This master's current role is: LEADER
W20260808 00:47:14.059708 21182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:47:14.059782 21182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:47:14.059845 21183 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:14.060072 21183 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:14.062065 21183 catalog_manager.cc:1347] Generated new cluster ID: 4b96e923d0484ca1a78d3eb08a628f21
I20260808 00:47:14.062119 21183 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:14.096792 21183 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:14.097096 21183 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:14.104558 21183 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd8468984fae48bab0a1c9eeb349a2e0: Generated new TSK 0
I20260808 00:47:14.104894 21183 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:47:14.109043 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21044
2026-08-08T00:47:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1657 ms)
[----------] 18 tests from AuthTokenIssuingTest (30315 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:14 dist-test-slave-kvfs krb5kdc[21193](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:14 dist-test-slave-kvfs krb5kdc[21193](info): set up 2 sockets
Aug 08 00:47:14 dist-test-slave-kvfs krb5kdc[21193](info): commencing operation
krb5kdc: starting...
W20260808 00:47:15.417201 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:15 dist-test-slave-kvfs krb5kdc[21193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:47:15.580447 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:35155
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:35155,127.20.105.125:33705,127.20.105.124:37149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:15.679458 21213 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:47:15.679742 21213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:15.679761 21213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:15.679783 21213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:15.681178 21213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:15.681229 21213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:15.681244 21213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:15.681257 21213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:15.681314 21213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:15.682933 21213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:35155,127.20.105.125:33705,127.20.105.124:37149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:35155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:15.683254 21213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:15.683529 21213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:15.689229 21223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:15.689232 21222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:15.689229 21225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:15.689628 21213 server_base.cc:1034] running on GCE node
I20260808 00:47:15.689918 21213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:15.690231 21213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:15.691385 21213 hybrid_clock.cc:648] HybridClock initialized: now 1786150035691367 us; error 29 us; skew 500 ppm
Aug 08 00:47:15 dist-test-slave-kvfs krb5kdc[21193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:15.694381 21213 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:47:15.694955 21213 webserver.cc:460] Webserver started at http://127.20.105.126:36363/ using document root <none> and password file <none>
I20260808 00:47:15.695230 21213 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:15.695303 21213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:15.695467 21213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:15.696846 21213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "5189c8dd32524f71b92e4e35d245492c"
format_stamp: "Formatted at 2026-08-08 00:47:15 on dist-test-slave-kvfs"
server_key: "ca0cd1f16004f247a4e9732721051907"
server_key_iv: "771d1b4753aef28589e4788eab28d07b"
server_key_version: "encryptionkey@0"
I20260808 00:47:15.697281 21213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "5189c8dd32524f71b92e4e35d245492c"
format_stamp: "Formatted at 2026-08-08 00:47:15 on dist-test-slave-kvfs"
server_key: "ca0cd1f16004f247a4e9732721051907"
server_key_iv: "771d1b4753aef28589e4788eab28d07b"
server_key_version: "encryptionkey@0"
I20260808 00:47:15.698859 21213 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:47:15.699747 21236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:15.699999 21213 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:15.700184 21213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "5189c8dd32524f71b92e4e35d245492c"
format_stamp: "Formatted at 2026-08-08 00:47:15 on dist-test-slave-kvfs"
server_key: "ca0cd1f16004f247a4e9732721051907"
server_key_iv: "771d1b4753aef28589e4788eab28d07b"
server_key_version: "encryptionkey@0"
I20260808 00:47:15.700278 21213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:15.720359 21213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:15.723151 21213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:15.723362 21213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:15.728842 21213 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:35155
I20260808 00:47:15.728902 21314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:35155 every 8 connection(s)
I20260808 00:47:15.729357 21213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:15.730548 21315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:15.732784 21315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:15.739132 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21213
I20260808 00:47:15.739269 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:15.739521 20901 external_mini_cluster.cc:1428] Setting key e026fbdb4a2ed86d8ec3590d0b2f332d
Aug 08 00:47:15 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:47:15.747205 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.733688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:35155 (local address 127.0.0.1:47404)
0808 00:47:15.733973 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.734029 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:15.734726 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:15.735084 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:15.735087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:15.735440 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:15.735646 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:15.735651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.736967 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.736967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:15.737718 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:15.737720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.738280 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.738281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:15.738353 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:15.739247 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:15.739262 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:15.741377 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:15.744114 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:15.744123 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:15.744126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:15.744457 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:15.745205 (+ 748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:15.745205 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:15.745206 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:15.745336 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:15.745873 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:15.745881 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:15.745992 (+ 111us) client_negotiation.cc:769] Sending connection context
0808 00:47:15.746130 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 00:47:15.746393 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":113,"threads_started":1}
I20260808 00:47:15.749279 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.733765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47404 (local address 127.20.105.126:35155)
0808 00:47:15.733987 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:15.733999 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:15.734174 (+ 175us) server_negotiation.cc:407] Connection header received
0808 00:47:15.734923 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:15.734926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:15.734956 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:15.735072 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:15.735765 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.736820 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.737816 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.738171 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.738311 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:15.741488 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:15.741499 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:15.741500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:15.741511 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:15.743931 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:15.744957 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:15.744961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:15.744962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:15.745125 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:15.745409 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:15.745410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:15.745410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:15.745783 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:15.745878 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:15.749053 (+ 3175us) server_negotiation.cc:299] Negotiation successful
0808 00:47:15.749174 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":118,"threads_started":1}
I20260808 00:47:15.750240 21315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:15.750911 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.750730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34724)
0808 00:47:15.750797 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.750875 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 00:47:15.751163 21239 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
I20260808 00:47:15.751787 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.751661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34732)
0808 00:47:15.751713 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.751746 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:47:15.752060 21315 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:33705: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.105.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab.
Entry for principal kudu/127.20.105.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab.
I20260808 00:47:15.783139 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.125:33705
--webserver_interface=127.20.105.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab
--principal=kudu/127.20.105.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:35155,127.20.105.125:33705,127.20.105.124:37149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:47:15.801378 21315 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } attempt: 1
I20260808 00:47:15.802078 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.801893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34742)
0808 00:47:15.801977 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.802027 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 00:47:15.802570 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.802433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34752)
0808 00:47:15.802490 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.802527 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 00:47:15.802696 21315 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:33705: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
I20260808 00:47:15.870880 21315 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } attempt: 2
I20260808 00:47:15.871613 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.871404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34756)
0808 00:47:15.871496 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.871558 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 00:47:15.872164 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.871997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34772)
0808 00:47:15.872106 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.872139 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260808 00:47:15.872305 21315 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:33705: Network error: Client connection negotiation failed: client connection to 127.20.105.125:33705: connect: Connection refused (error 111)
W20260808 00:47:15.914830 21326 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:47:15.915184 21326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:15.915220 21326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:15.915267 21326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:15.916781 21326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:15.916867 21326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:15.916894 21326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:15.916929 21326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:15.916958 21326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:15.918826 21326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:35155,127.20.105.125:33705,127.20.105.124:37149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab
--principal=kudu/127.20.105.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.125:33705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.105.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:15.919214 21326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:15.919662 21326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:15.926285 21336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:15.926268 21338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:15.926268 21334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:15.926659 21326 server_base.cc:1034] running on GCE node
I20260808 00:47:15.926994 21326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:15.927332 21326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:15.928536 21326 hybrid_clock.cc:648] HybridClock initialized: now 1786150035928512 us; error 49 us; skew 500 ppm
Aug 08 00:47:15 dist-test-slave-kvfs krb5kdc[21193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:15.931730 21326 init.cc:377] Logged in from keytab as kudu/127.20.105.125@KRBTEST.COM (short username kudu)
I20260808 00:47:15.932379 21326 webserver.cc:460] Webserver started at http://127.20.105.125:41765/ using document root <none> and password file <none>
I20260808 00:47:15.932693 21326 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:15.932773 21326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:15.932950 21326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:15.934303 21326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/data/instance:
uuid: "c08835dd6fd84487a79ae57e287c2882"
format_stamp: "Formatted at 2026-08-08 00:47:15 on dist-test-slave-kvfs"
server_key: "29ae429a7b42aad4cea631da29bdf901"
server_key_iv: "b295623d7f1c4ec3d5e6d0328cc53098"
server_key_version: "encryptionkey@0"
I20260808 00:47:15.934769 21326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/wal/instance:
uuid: "c08835dd6fd84487a79ae57e287c2882"
format_stamp: "Formatted at 2026-08-08 00:47:15 on dist-test-slave-kvfs"
server_key: "29ae429a7b42aad4cea631da29bdf901"
server_key_iv: "b295623d7f1c4ec3d5e6d0328cc53098"
server_key_version: "encryptionkey@0"
I20260808 00:47:15.936466 21326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:47:15.937412 21348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:15.937664 21326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:15.937793 21326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/wal
uuid: "c08835dd6fd84487a79ae57e287c2882"
format_stamp: "Formatted at 2026-08-08 00:47:15 on dist-test-slave-kvfs"
server_key: "29ae429a7b42aad4cea631da29bdf901"
server_key_iv: "b295623d7f1c4ec3d5e6d0328cc53098"
server_key_version: "encryptionkey@0"
I20260808 00:47:15.937886 21326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:15.950588 21326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:15.953733 21326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:15.953933 21326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:15.959331 21326 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.125:33705
I20260808 00:47:15.959364 21427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.125:33705 every 8 connection(s)
I20260808 00:47:15.959870 21326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
I20260808 00:47:15.961174 21429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:15.963155 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21326
I20260808 00:47:15.963317 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-1/wal/instance
I20260808 00:47:15.963485 21315 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } attempt: 3
I20260808 00:47:15.963619 20901 external_mini_cluster.cc:1428] Setting key 038468b0516880fee48c1bf00397d32b
I20260808 00:47:15.963677 21429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:47:15 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
Aug 08 00:47:15 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:47:15.975958 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.964711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.20.105.126:35155)
0808 00:47:15.964768 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:15.964771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:15.965024 (+ 253us) server_negotiation.cc:407] Connection header received
0808 00:47:15.965562 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:15.965564 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:15.965585 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:15.965646 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:15.966521 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.967706 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.969029 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.969380 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.969450 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:15.972655 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:15.972656 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:15.972657 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:15.972672 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:15.974297 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:15.974794 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:15.974795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:15.974796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:15.974878 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:15.975159 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:15.975161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:15.975161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:15.975318 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:15.975396 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:15.975737 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 00:47:15.975842 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":31}
WARNING: no policy specified for kudu/127.20.105.124@KRBTEST.COM; defaulting to no policy
I20260808 00:47:15.976236 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.964651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:35155 (local address 127.0.0.1:47408)
0808 00:47:15.964845 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.964906 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:15.965419 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:15.965669 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:15.965674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:15.966048 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:15.966401 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:15.966408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.967823 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.967824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:15.968911 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:15.968914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.969462 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.969463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:15.969576 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:15.970152 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:15.970162 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:15.972544 (+ 2382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:15.974396 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:15.974405 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:15.974408 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:15.974707 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:15.974958 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:15.974958 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:15.974959 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:15.975081 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:15.975400 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:15.975409 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:15.975582 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 00:47:15.975701 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 00:47:15.975893 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":3,"thread_start_us":99,"threads_started":1}
I20260808 00:47:15.976392 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.964226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34776 (local address 127.20.105.125:33705)
0808 00:47:15.964481 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:15.964495 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:15.964554 (+ 59us) server_negotiation.cc:407] Connection header received
0808 00:47:15.965385 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:15.965397 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:15.965429 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:15.966047 (+ 618us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:15.966147 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.967660 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.968618 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.969142 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.969267 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:15.971891 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:15.971903 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:15.971904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:15.971913 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:15.974393 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:15.974997 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:15.975001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:15.975002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:15.975094 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:15.975367 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:15.975367 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:15.975368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:15.975632 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:15.976081 (+ 449us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:15.976137 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 00:47:15.976227 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":186,"thread_start_us":108,"threads_started":1}
I20260808 00:47:15.976635 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.963956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34776)
0808 00:47:15.964088 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.964104 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:15.964162 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:15.965581 (+ 1419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:15.965583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:15.965844 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:15.965945 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:15.965948 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.967844 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.967846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:15.968507 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:15.968509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.969241 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.969242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:15.969347 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:15.969948 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:15.969956 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:15.971781 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:15.974486 (+ 2705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:15.974487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:15.974487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:15.974698 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:15.975193 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:15.975195 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:15.975195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:15.975278 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:15.975725 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:15.975727 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:15.975788 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 00:47:15.975847 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:47:15.975956 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 00:47:15.977383 21429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:15.977519 21315 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:15.977890 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.977769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51454)
0808 00:47:15.977821 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.977860 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:47:15.978292 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.978163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51466)
0808 00:47:15.978223 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.978260 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 00:47:15.978386 21315 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37149: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.124@KRBTEST.COM" created.
Aug 08 00:47:15 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:47:15.985558 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.977765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.20.105.125:33705)
0808 00:47:15.977807 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:15.977809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:15.977845 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:47:15.977911 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:15.977912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:15.977924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:15.977985 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:15.978367 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.978977 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.979974 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.980375 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.980458 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:15.982762 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:15.982764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:15.982764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:15.982772 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:15.984307 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:15.984672 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:15.984674 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:15.984674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:15.984730 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:15.984931 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:15.984931 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:15.984932 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:15.985055 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:15.985134 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:15.985297 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 00:47:15.985423 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":18}
I20260808 00:47:15.985565 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.977720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34782)
0808 00:47:15.977802 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.977813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:15.977858 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:15.978025 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:15.978026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:15.978204 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:15.978264 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:15.978267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.979084 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.979085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:15.979871 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:15.979875 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:15.980464 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:15.980465 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:15.980528 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:15.981026 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:15.981031 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:15.982658 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:15.984409 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:15.984410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:15.984410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:15.984586 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:15.984797 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:15.984798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:15.984799 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:15.984857 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:15.985123 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:15.985123 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:15.985168 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 00:47:15.985270 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 00:47:15.985477 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":95}
I20260808 00:47:15.985949 21429 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:15.986485 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.986258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51482)
0808 00:47:15.986336 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.986433 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 00:47:15.986729 21354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
I20260808 00:47:15.987347 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:15.987195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51484)
0808 00:47:15.987265 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:15.987307 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 00:47:15.987521 21429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37149: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab.
Entry for principal kudu/127.20.105.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab.
I20260808 00:47:16.003698 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.124:37149
--webserver_interface=127.20.105.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab
--principal=kudu/127.20.105.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:39041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:35155,127.20.105.125:33705,127.20.105.124:37149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:47:16.009594 21315 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } attempt: 1
I20260808 00:47:16.010272 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.010049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51488)
0808 00:47:16.010137 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.010197 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 00:47:16.010742 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.010612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51502)
0808 00:47:16.010672 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.010708 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 00:47:16.010897 21315 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37149: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
I20260808 00:47:16.036904 21429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } attempt: 1
I20260808 00:47:16.037741 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.037537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51508)
0808 00:47:16.037623 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.037689 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 00:47:16.038304 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.038193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51512)
0808 00:47:16.038247 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.038280 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:47:16.038445 21429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37149: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
I20260808 00:47:16.086087 21315 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } attempt: 2
I20260808 00:47:16.086787 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.086622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51518)
0808 00:47:16.086699 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.086747 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 00:47:16.087302 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.087201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51532)
0808 00:47:16.087251 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.087280 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 00:47:16.087440 21315 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37149: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
W20260808 00:47:16.106071 21438 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:47:16.106395 21438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:16.106415 21438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:16.106441 21438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 00:47:16.106629 21429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } attempt: 2
I20260808 00:47:16.107314 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.107110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51546)
0808 00:47:16.107207 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.107264 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 00:47:16.107792 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.107675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51562)
0808 00:47:16.107729 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.107765 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 00:47:16.107961 21429 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37149: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
W20260808 00:47:16.108315 21438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:16.108382 21438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:16.108409 21438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:16.108438 21438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:16.108466 21438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:16.110337 21438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:39041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:35155,127.20.105.125:33705,127.20.105.124:37149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab
--principal=kudu/127.20.105.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.124:37149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.105.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:16.110687 21438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:16.110972 21438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:16.117313 21448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:16.117254 21444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:16.117455 21438 server_base.cc:1034] running on GCE node
W20260808 00:47:16.117296 21445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:16.117818 21438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:16.118152 21438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:16.119316 21438 hybrid_clock.cc:648] HybridClock initialized: now 1786150036119287 us; error 50 us; skew 500 ppm
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150036, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:16.122637 21438 init.cc:377] Logged in from keytab as kudu/127.20.105.124@KRBTEST.COM (short username kudu)
I20260808 00:47:16.123149 21438 webserver.cc:460] Webserver started at http://127.20.105.124:33487/ using document root <none> and password file <none>
I20260808 00:47:16.123368 21438 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:16.123412 21438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:16.123520 21438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:16.124835 21438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/data/instance:
uuid: "7b1e8259f668495b8aa071a49118ab7a"
format_stamp: "Formatted at 2026-08-08 00:47:16 on dist-test-slave-kvfs"
server_key: "f8c3d32ca0c0dd30a318dbe9d86d4f02"
server_key_iv: "02b8f8344f1b4b516f6542bb555b41c6"
server_key_version: "encryptionkey@0"
I20260808 00:47:16.125208 21438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/wal/instance:
uuid: "7b1e8259f668495b8aa071a49118ab7a"
format_stamp: "Formatted at 2026-08-08 00:47:16 on dist-test-slave-kvfs"
server_key: "f8c3d32ca0c0dd30a318dbe9d86d4f02"
server_key_iv: "02b8f8344f1b4b516f6542bb555b41c6"
server_key_version: "encryptionkey@0"
I20260808 00:47:16.127028 21438 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260808 00:47:16.127846 21459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:16.128065 21438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:16.128180 21438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/wal
uuid: "7b1e8259f668495b8aa071a49118ab7a"
format_stamp: "Formatted at 2026-08-08 00:47:16 on dist-test-slave-kvfs"
server_key: "f8c3d32ca0c0dd30a318dbe9d86d4f02"
server_key_iv: "02b8f8344f1b4b516f6542bb555b41c6"
server_key_version: "encryptionkey@0"
I20260808 00:47:16.128252 21438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:16.186656 21315 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } attempt: 3
I20260808 00:47:16.187495 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.187274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51576)
0808 00:47:16.187402 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.187449 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 00:47:16.187991 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.187865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51582)
0808 00:47:16.187927 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.187963 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 00:47:16.188138 21315 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:37149: Network error: Client connection negotiation failed: client connection to 127.20.105.124:37149: connect: Connection refused (error 111)
I20260808 00:47:16.188862 21438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:16.191483 21438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:16.191680 21438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:16.196751 21438 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.124:37149
I20260808 00:47:16.196789 21545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.124:37149 every 8 connection(s)
I20260808 00:47:16.197247 21438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
I20260808 00:47:16.198369 21546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:16.199198 21429 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } attempt: 3
I20260808 00:47:16.201188 21546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:16.203502 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21438
I20260808 00:47:16.203629 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149729697709-20901-0/minicluster-data/master-2/wal/instance
I20260808 00:47:16.203855 20901 external_mini_cluster.cc:1428] Setting key d2e9f9068aeaf71a8932f1c3f2476528
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150036, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:47:16.211498 21434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.199652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51588)
0808 00:47:16.199731 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.199745 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:16.199800 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:16.201240 (+ 1440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:16.201242 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:16.201432 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:16.201498 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.201501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.203259 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.203260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.203961 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.203964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.205062 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.205063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.205128 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.205693 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:16.205700 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:16.207496 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:16.209916 (+ 2420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.209917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.209918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.210189 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.210486 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.210487 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.210487 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.210564 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.211096 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:16.211097 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:16.211160 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 00:47:16.211215 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:47:16.211354 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 00:47:16.211953 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.202108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.20.105.126:35155)
0808 00:47:16.202180 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.202182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.202344 (+ 162us) server_negotiation.cc:407] Connection header received
0808 00:47:16.202394 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.202395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.202406 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.202453 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.203092 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.203978 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.205239 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.205928 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.206009 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.208664 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.208666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.208667 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.208676 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.210349 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.210817 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.210818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.210818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.210882 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.211150 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.211151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.211152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.211313 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.211406 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.211715 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.211820 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:47:16.212332 21549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.201977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:35155 (local address 127.0.0.1:47414)
0808 00:47:16.202199 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.202248 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:16.202334 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:16.202481 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:16.202483 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:16.202879 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:16.203009 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.203016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.204140 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.204143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.205109 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.205112 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.206028 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.206029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.206118 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.206694 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:16.206704 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:16.208582 (+ 1878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:16.210440 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.210446 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.210449 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.210721 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.210952 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.210953 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.210953 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.211072 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.211439 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:16.211446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:16.211534 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 00:47:16.212056 (+ 522us) client_negotiation.cc:240] Negotiation successful
0808 00:47:16.212197 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":138,"threads_started":1}
I20260808 00:47:16.212345 21548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.199873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.20.105.124:37149)
0808 00:47:16.200247 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.200258 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.200300 (+ 42us) server_negotiation.cc:407] Connection header received
0808 00:47:16.200939 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.200950 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.201026 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.201229 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.201635 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.203113 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.204100 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.204973 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.205104 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.207614 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.207622 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.207622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.207630 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.209819 (+ 2189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.210291 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.210295 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.210296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.210420 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.210644 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.210645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.210645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.211003 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.211778 (+ 775us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.211838 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.212011 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":243,"threads_started":1}
I20260808 00:47:16.212991 21546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:16.214769 21429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882: Bootstrap starting.
I20260808 00:47:16.215054 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.204507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51604 (local address 127.20.105.124:37149)
0808 00:47:16.204663 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.204664 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.204673 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:47:16.204693 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.204694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.204704 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.204785 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.205591 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.206169 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.207298 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.207589 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.207678 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.210920 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.210922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.210922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.210929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.213447 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.213809 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.213810 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.213811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.213897 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.214117 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.214118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.214118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.214236 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.214388 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.214768 (+ 380us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.214922 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":133,"thread_start_us":79,"threads_started":1}
I20260808 00:47:16.216436 21429 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:16.216853 21429 log.cc:824] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882: Log is configured to *not* fsync() on all Append() calls
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150036, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:47:16.217865 21429 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882: No bootstrap required, opened a new log
I20260808 00:47:16.220093 21429 raft_consensus.cc:348] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } }
I20260808 00:47:16.220301 21429 raft_consensus.cc:729] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08835dd6fd84487a79ae57e287c2882, State: Initialized, Role: FOLLOWER
I20260808 00:47:16.220413 21429 consensus_queue.cc:260] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } }
I20260808 00:47:16.220800 21554 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.220911 21429 sys_catalog.cc:564] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:16.221012 21554 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:16.221626 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.213390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34790 (local address 127.20.105.125:33705)
0808 00:47:16.213434 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.213436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.213444 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:47:16.213469 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.213470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.213482 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.213549 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.213870 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.214596 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.215618 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.215813 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.215876 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.218198 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.218200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.218201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.218208 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.219988 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.220672 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.220675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.220675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.220745 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.220970 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.220971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.220971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.221140 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.221215 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.221429 (+ 214us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.221524 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:47:16.221659 21549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.213320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:33705 (local address 127.0.0.1:34790)
0808 00:47:16.213369 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.213380 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:16.213417 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:16.213601 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:16.213602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:16.213744 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:16.213786 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.213788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.214704 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.214705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.215507 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.215512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.215897 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.215898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.215975 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.216460 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:16.216466 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:16.218101 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:16.220098 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.220099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.220100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.220326 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.220831 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.220832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.220833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.220898 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.221247 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:16.221248 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:16.221297 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 00:47:16.221368 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:47:16.221489 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 00:47:16.222344 21546 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
W20260808 00:47:16.223659 21567 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:47:16.223763 21567 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:47:16.226883 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.216894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.20.105.126:35155)
0808 00:47:16.216999 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.217001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.217009 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:47:16.217033 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.217034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.217074 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.217128 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.217938 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.218630 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.219704 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.220071 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.220116 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.223308 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.223310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.223311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.223320 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.225328 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.225768 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.225770 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.225771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.225843 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.226067 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.226069 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.226069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.226224 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.226408 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.226660 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.226763 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150036, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:47:16.224160 21326 master_runner.cc:186] Error getting master registration for 127.20.105.126:35155: OK, instance_id { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" instance_seqno: 1786150035727268 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:47:16.229417 21326 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:47:16.230984 21549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.222660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51620)
0808 00:47:16.222755 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.222767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:16.222811 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:16.222992 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:16.222993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:16.223173 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:16.223215 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.223217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.224176 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.224177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.225071 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.225075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.225718 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.225719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.225774 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.226233 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:16.226239 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:16.227820 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:16.229783 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.229784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.229785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.229971 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.230195 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.230195 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.230196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.230255 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.230582 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:16.230582 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:16.230625 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:47:16.230717 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:47:16.230851 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 00:47:16.230988 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.222763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51620 (local address 127.20.105.124:37149)
0808 00:47:16.222805 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.222806 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.222816 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:47:16.222893 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.222895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.222904 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.222948 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.223320 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.224097 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.225268 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.225585 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.225664 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.227926 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.227927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.227928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.227935 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.229677 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.230060 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.230061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.230062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.230117 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.230326 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.230327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.230327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.230508 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.230591 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.230776 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.230903 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":23}
I20260808 00:47:16.233541 21546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a: Bootstrap starting.
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:47:16.234683 21546 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:16.235173 21546 log.cc:824] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:16.236690 21546 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a: No bootstrap required, opened a new log
I20260808 00:47:16.238511 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.227950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34804 (local address 127.20.105.125:33705)
0808 00:47:16.228000 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.228002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.228012 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:47:16.228148 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.228149 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.228163 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.228196 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.229046 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.229801 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.230984 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.231321 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.231415 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.235233 (+ 3818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.235235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.235236 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.235247 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.237157 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.237557 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.237559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.237559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.237632 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.237871 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.237872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.237872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.238021 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.238248 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.238288 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.238399 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:47:16.238637 21546 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } }
I20260808 00:47:16.238891 21546 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b1e8259f668495b8aa071a49118ab7a, State: Initialized, Role: FOLLOWER
I20260808 00:47:16.239029 21546 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } }
I20260808 00:47:16.239513 21546 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:16.239414 21570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.239601 21570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:16.240499 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 00:47:16.242933 21584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:47:16.243006 21584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:47:16.242202 21282 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5189c8dd32524f71b92e4e35d245492c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47422
I20260808 00:47:16.242657 21438 master_runner.cc:186] Error getting master registration for 127.20.105.126:35155: OK, instance_id { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" instance_seqno: 1786150035727268 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:47:16.245697 21438 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:47:16.352306 21315 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } attempt: 4
Aug 08 00:47:16 dist-test-slave-kvfs krb5kdc[21193](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150035, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:47:16.362725 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.352961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:37149 (local address 127.0.0.1:51628)
0808 00:47:16.353065 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:16.353085 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:16.353142 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:16.353536 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:16.353538 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:16.353850 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:16.353936 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.353940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.354994 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.354996 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.355811 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:16.355816 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.356420 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.356422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:16.356493 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.357176 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:16.357184 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:16.359152 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:16.361378 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.361380 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.361381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.361656 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.361904 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:16.361905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:16.361905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:16.361975 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:16.362312 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:16.362314 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:16.362375 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 00:47:16.362445 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:47:16.362594 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 00:47:16.363466 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.353237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51628 (local address 127.20.105.124:37149)
0808 00:47:16.353320 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.353329 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.353343 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:47:16.353387 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.353388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.353403 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.353503 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.354052 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.354864 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.355936 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.356256 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.356335 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.359273 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.359275 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.359276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.359287 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.361259 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.361755 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.361757 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.361757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.361828 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.362046 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.362047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.362047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.362212 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.362281 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.363262 (+ 981us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.363368 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:47:16.366258 21315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c: Bootstrap starting.
I20260808 00:47:16.367321 21315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:16.367758 21315 log.cc:824] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:16.368945 21315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c: No bootstrap required, opened a new log
I20260808 00:47:16.371070 21315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } }
I20260808 00:47:16.371423 21315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5189c8dd32524f71b92e4e35d245492c, State: Initialized, Role: FOLLOWER
I20260808 00:47:16.371707 21315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } }
I20260808 00:47:16.372457 21587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.372574 21587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:16.372766 21282 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 00:47:16.372848 21282 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:16.374161 21282 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } }
I20260808 00:47:16.375002 21391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5189c8dd32524f71b92e4e35d245492c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c08835dd6fd84487a79ae57e287c2882"
I20260808 00:47:16.375006 21502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5189c8dd32524f71b92e4e35d245492c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7b1e8259f668495b8aa071a49118ab7a"
I20260808 00:47:16.375172 21391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:16.375172 21502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:16.376386 21391 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5189c8dd32524f71b92e4e35d245492c in term 1.
I20260808 00:47:16.376680 21502 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5189c8dd32524f71b92e4e35d245492c in term 1.
I20260808 00:47:16.376786 21239 leader_election.cc:302] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5189c8dd32524f71b92e4e35d245492c, c08835dd6fd84487a79ae57e287c2882; no voters:
I20260808 00:47:16.376947 21587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:16.377143 21587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [term 1 LEADER]: Becoming Leader. State: Replica: 5189c8dd32524f71b92e4e35d245492c, State: Running, Role: LEADER
I20260808 00:47:16.377161 21282 leader_election.cc:288] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [CANDIDATE]: Term 1 election: Requested vote from peers c08835dd6fd84487a79ae57e287c2882 (127.20.105.125:33705), 7b1e8259f668495b8aa071a49118ab7a (127.20.105.124:37149)
I20260808 00:47:16.377254 21587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } }
I20260808 00:47:16.377797 21587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5189c8dd32524f71b92e4e35d245492c. Latest consensus state: current_term: 1 leader_uuid: "5189c8dd32524f71b92e4e35d245492c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.377882 21587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:16.378722 21315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:16.384435 21594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:16.385385 21594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:16.387765 21502 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [term 1 FOLLOWER]: Refusing update from remote peer 5189c8dd32524f71b92e4e35d245492c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:47:16.388346 21588 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:47:16.390194 21570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5189c8dd32524f71b92e4e35d245492c. Latest consensus state: current_term: 1 leader_uuid: "5189c8dd32524f71b92e4e35d245492c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.390327 21570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:16.390586 21391 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [term 1 FOLLOWER]: Refusing update from remote peer 5189c8dd32524f71b92e4e35d245492c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:47:16.391031 21588 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:47:16.392865 21554 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5189c8dd32524f71b92e4e35d245492c. Latest consensus state: current_term: 1 leader_uuid: "5189c8dd32524f71b92e4e35d245492c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.393013 21554 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:16.397034 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.379117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51642 (local address 127.20.105.124:37149)
0808 00:47:16.379166 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.379168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.379682 (+ 514us) server_negotiation.cc:407] Connection header received
0808 00:47:16.379797 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.379798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.379811 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.379843 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.380811 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.381605 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.382739 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.383033 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.383077 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.393714 (+ 10637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.393717 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.393717 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.393727 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.395636 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.396112 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.396114 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.396114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.396174 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.396359 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.396360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.396360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.396519 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.396607 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.396815 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.396916 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:47:16.401690 21320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.378810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47436 (local address 127.20.105.126:35155)
0808 00:47:16.378856 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.378859 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.383858 (+ 4999us) server_negotiation.cc:407] Connection header received
0808 00:47:16.383966 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.383968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.383980 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.384011 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.384900 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.385642 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.386670 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.386975 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.387020 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.398407 (+ 11387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.398410 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.398420 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.398432 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.400188 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.400644 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.400646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.400646 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.400718 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.400933 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.400934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.400934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.401109 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.401224 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.401469 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.401582 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:47:16.401954 21623 mvcc.cc:205] Tried to move back new op lower bound from 7316070549040406528 to 7316070549002330112. Current Snapshot: MvccSnapshot[applied={T|T < 7316070549040406528}]
I20260808 00:47:16.402098 21588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5189c8dd32524f71b92e4e35d245492c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.402247 21588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:16.403160 21570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5189c8dd32524f71b92e4e35d245492c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.403294 21570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:16.405123 21588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5189c8dd32524f71b92e4e35d245492c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.405247 21588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:16.406250 21554 sys_catalog.cc:455] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5189c8dd32524f71b92e4e35d245492c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5189c8dd32524f71b92e4e35d245492c" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 35155 } } peers { permanent_uuid: "c08835dd6fd84487a79ae57e287c2882" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 33705 } } peers { permanent_uuid: "7b1e8259f668495b8aa071a49118ab7a" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 37149 } } }
I20260808 00:47:16.406400 21554 sys_catalog.cc:458] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:16.406438 21624 mvcc.cc:205] Tried to move back new op lower bound from 7316070549040406528 to 7316070549002330112. Current Snapshot: MvccSnapshot[applied={T|T < 7316070549040406528}]
I20260808 00:47:16.406615 21628 mvcc.cc:205] Tried to move back new op lower bound from 7316070549040406528 to 7316070549002330112. Current Snapshot: MvccSnapshot[applied={T|T < 7316070549040406528}]
I20260808 00:47:16.407437 21433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:16.379366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34812 (local address 127.20.105.125:33705)
0808 00:47:16.379423 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.379425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.387825 (+ 8400us) server_negotiation.cc:407] Connection header received
0808 00:47:16.387952 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.387954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.387974 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.388010 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.388936 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.389652 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.391297 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:16.391604 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:16.391648 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:16.403693 (+ 12045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:16.403696 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:16.403697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:16.403723 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:16.405719 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.406255 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.406257 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.406257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.406349 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:16.406770 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:16.406771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:16.406771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:16.406929 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:16.407021 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:16.407225 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 00:47:16.407320 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:47:16.410338 21594 catalog_manager.cc:1347] Generated new cluster ID: f9571f3703f04525a517c5b049132420
I20260808 00:47:16.410422 21594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:16.428833 21594 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:16.429585 21594 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:16.441476 21594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5189c8dd32524f71b92e4e35d245492c: Generated new TSK 0
I20260808 00:47:16.441998 21594 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:47:16.469403 21551 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.468217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51654 (local address 127.20.105.124:37149)
0808 00:47:16.468283 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.468285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.468446 (+ 161us) server_negotiation.cc:407] Connection header received
0808 00:47:16.468586 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.468587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.468604 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.468642 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.469315 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51654: BlockingRecv error: recv got EOF from 127.0.0.1:51654 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 00:47:16.469410 21320 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.468219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47452 (local address 127.20.105.126:35155)
0808 00:47:16.468278 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.468281 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.468574 (+ 293us) server_negotiation.cc:407] Connection header received
0808 00:47:16.468688 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.468689 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.468703 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.468736 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.469290 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47452: BlockingRecv error: recv got EOF from 127.0.0.1:47452 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260808 00:47:16.470172 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.468636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.20.105.125:33705)
0808 00:47:16.468687 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.468689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.469071 (+ 382us) server_negotiation.cc:407] Connection header received
0808 00:47:16.469185 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.469186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.469201 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.469248 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.470075 (+ 827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34828: BlockingRecv error: recv got EOF from 127.0.0.1:34828 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260808 00:47:16.472007 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.471430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.20.105.125:33705)
0808 00:47:16.471467 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.471469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.471515 (+ 46us) server_negotiation.cc:407] Connection header received
0808 00:47:16.471587 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.471588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.471611 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.471683 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.471950 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34834: BlockingRecv error: recv got EOF from 127.0.0.1:34834 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 00:47:16.474768 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.474214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34850 (local address 127.20.105.125:33705)
0808 00:47:16.474276 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.474278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.474292 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:47:16.474313 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.474314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.474323 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.474388 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.474706 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34850: BlockingRecv error: recv got EOF from 127.0.0.1:34850 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 00:47:16.478448 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.477776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34862 (local address 127.20.105.125:33705)
0808 00:47:16.477836 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.477838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.477928 (+ 90us) server_negotiation.cc:407] Connection header received
0808 00:47:16.478010 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.478011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.478029 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.478094 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.478393 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34862: BlockingRecv error: recv got EOF from 127.0.0.1:34862 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 00:47:16.484499 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.483634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34864 (local address 127.20.105.125:33705)
0808 00:47:16.483708 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.483710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.483763 (+ 53us) server_negotiation.cc:407] Connection header received
0808 00:47:16.483829 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.483830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.483844 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.483903 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.484429 (+ 526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34864: BlockingRecv error: recv got EOF from 127.0.0.1:34864 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 00:47:16.494964 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.494013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.20.105.125:33705)
0808 00:47:16.494087 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.494089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.494210 (+ 121us) server_negotiation.cc:407] Connection header received
0808 00:47:16.494327 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.494328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.494350 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.494422 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.494850 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34878: BlockingRecv error: recv got EOF from 127.0.0.1:34878 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 00:47:16.513646 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.512694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34894 (local address 127.20.105.125:33705)
0808 00:47:16.512773 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.512776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.512892 (+ 116us) server_negotiation.cc:407] Connection header received
0808 00:47:16.512964 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.512966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.512983 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.513081 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.513549 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34894: BlockingRecv error: recv got EOF from 127.0.0.1:34894 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 00:47:16.547989 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.547099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34900 (local address 127.20.105.125:33705)
0808 00:47:16.547172 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.547175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.547275 (+ 100us) server_negotiation.cc:407] Connection header received
0808 00:47:16.547328 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.547330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.547347 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.547446 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.547896 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34900: BlockingRecv error: recv got EOF from 127.0.0.1:34900 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 00:47:16.614609 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.613545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34906 (local address 127.20.105.125:33705)
0808 00:47:16.613663 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.613666 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.613785 (+ 119us) server_negotiation.cc:407] Connection header received
0808 00:47:16.613937 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.613939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.613956 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.614053 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.614507 (+ 454us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34906: BlockingRecv error: recv got EOF from 127.0.0.1:34906 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 00:47:16.745124 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:16.744082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34918 (local address 127.20.105.125:33705)
0808 00:47:16.744188 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:16.744192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:16.744343 (+ 151us) server_negotiation.cc:407] Connection header received
0808 00:47:16.744470 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:16.744472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:16.744492 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:16.744575 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:16.745022 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34918: BlockingRecv error: recv got EOF from 127.0.0.1:34918 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 00:47:17.003713 21433 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:17.002822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34924 (local address 127.20.105.125:33705)
0808 00:47:17.002897 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:17.002899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:17.002914 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:47:17.003016 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:17.003017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:17.003031 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:17.003102 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:17.003598 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34924: BlockingRecv error: recv got EOF from 127.0.0.1:34924 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:47:17.224538 21567 catalog_manager.cc:1259] Loaded cluster ID: f9571f3703f04525a517c5b049132420
I20260808 00:47:17.224697 21567 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882: loading cluster ID for follower catalog manager: success
I20260808 00:47:17.225749 21567 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882: acquiring CA information for follower catalog manager: success
I20260808 00:47:17.226181 21567 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c08835dd6fd84487a79ae57e287c2882: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:47:17.243767 21584 catalog_manager.cc:1259] Loaded cluster ID: f9571f3703f04525a517c5b049132420
I20260808 00:47:17.243896 21584 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a: loading cluster ID for follower catalog manager: success
I20260808 00:47:17.245620 21584 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a: acquiring CA information for follower catalog manager: success
I20260808 00:47:17.246132 21584 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b1e8259f668495b8aa071a49118ab7a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:47:17.520516 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:17.517202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34932 (local address 127.20.105.125:33705)
0808 00:47:17.517386 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:17.517388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:17.517402 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:47:17.517489 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:17.517490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:17.517577 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:17.517688 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:17.517849 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.518567 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.519420 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.519655 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.519718 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:17.520000 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:17.520241 (+ 241us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:17.520325 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:17.520377 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 00:47:17.520402 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":109,"threads_started":1}
I20260808 00:47:17.525808 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:17.523079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51664 (local address 127.20.105.124:37149)
0808 00:47:17.523219 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:17.523221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:17.523229 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:47:17.523284 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:17.523285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:17.523334 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:17.523431 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:17.523537 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.524125 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.524929 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.525151 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.525213 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:17.525374 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:17.525517 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:17.525584 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:17.525677 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 00:47:17.525703 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260808 00:47:17.530864 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:17.527949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51666 (local address 127.20.105.124:37149)
0808 00:47:17.528004 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:17.528006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:17.528088 (+ 82us) server_negotiation.cc:407] Connection header received
0808 00:47:17.528177 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:17.528178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:17.528188 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:17.528266 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:17.528398 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.529044 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.530031 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.530340 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.530420 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:17.530577 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:17.530620 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:17.530670 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:17.530729 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 00:47:17.530756 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:47:17.531462 21737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:17.528086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.20.105.125:33705)
0808 00:47:17.528130 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:17.528132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:17.528490 (+ 358us) server_negotiation.cc:407] Connection header received
0808 00:47:17.528546 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:17.528547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:17.528564 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:17.528620 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:17.528754 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.529495 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.530413 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.530728 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.530791 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:17.531140 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:17.531182 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:17.531228 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:17.531336 (+ 108us) server_negotiation.cc:299] Negotiation successful
0808 00:47:17.531370 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:47:17.531589 21757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:17.527853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47462 (local address 127.20.105.126:35155)
0808 00:47:17.528147 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:17.528149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:17.528159 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:47:17.528226 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:17.528227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:17.528308 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:17.528424 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:17.528579 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.529393 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.530333 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:17.530655 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:17.530720 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:17.531015 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:17.531184 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:17.531253 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:17.531445 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 00:47:17.531482 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":70,"threads_started":1}
I20260808 00:47:17.539716 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21213
I20260808 00:47:17.563169 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21326
I20260808 00:47:17.583815 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21438
2026-08-08T00:47:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3469 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:47:17 dist-test-slave-kvfs krb5kdc[21763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:47:17 dist-test-slave-kvfs krb5kdc[21763](info): set up 2 sockets
Aug 08 00:47:17 dist-test-slave-kvfs krb5kdc[21763](info): commencing operation
krb5kdc: starting...
W20260808 00:47:18.865665 20901 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/test-user.keytab.
Aug 08 00:47:18 dist-test-slave-kvfs krb5kdc[21763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.105.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
Entry for principal kudu/127.20.105.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab.
I20260808 00:47:19.015357 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:43579
--webserver_interface=127.20.105.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:38067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:43579,127.20.105.125:35253,127.20.105.124:38263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:47:19.112409 21784 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:47:19.112735 21784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:19.112759 21784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:19.112815 21784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:19.114259 21784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:19.114319 21784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:19.114349 21784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:19.114396 21784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:19.114439 21784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:19.116394 21784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:38067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:43579,127.20.105.125:35253,127.20.105.124:38263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.126.keytab
--principal=kudu/127.20.105.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.126:43579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.105.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:19.116776 21784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:19.117125 21784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:19.123538 21799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:19.123548 21793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:19.123544 21795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:19.124182 21784 server_base.cc:1034] running on GCE node
I20260808 00:47:19.124605 21784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:19.125047 21784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:19.126256 21784 hybrid_clock.cc:648] HybridClock initialized: now 1786150039126236 us; error 51 us; skew 500 ppm
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:19.129731 21784 init.cc:377] Logged in from keytab as kudu/127.20.105.126@KRBTEST.COM (short username kudu)
I20260808 00:47:19.130317 21784 webserver.cc:460] Webserver started at http://127.20.105.126:34319/ using document root <none> and password file <none>
I20260808 00:47:19.130599 21784 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:19.130673 21784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:19.130837 21784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:19.132261 21784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/data/instance:
uuid: "764cf92450844d4083faa765eee45470"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "71062c9b2f8e188b1ae06c9f5c7c5c98"
server_key_iv: "39f34cadef38adfbbd194839ff7c672d"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.132692 21784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/wal/instance:
uuid: "764cf92450844d4083faa765eee45470"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "71062c9b2f8e188b1ae06c9f5c7c5c98"
server_key_iv: "39f34cadef38adfbbd194839ff7c672d"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.134230 21784 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:19.135120 21808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:19.135326 21784 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:19.135444 21784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/wal
uuid: "764cf92450844d4083faa765eee45470"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "71062c9b2f8e188b1ae06c9f5c7c5c98"
server_key_iv: "39f34cadef38adfbbd194839ff7c672d"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.135533 21784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:19.157505 21784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:19.160552 21784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:19.160749 21784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:19.166234 21784 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.126:43579
I20260808 00:47:19.166296 21901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.126:43579 every 8 connection(s)
I20260808 00:47:19.166752 21784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/data/info.pb
I20260808 00:47:19.167950 21902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:19.169948 21902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:19.172698 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21784
I20260808 00:47:19.172827 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-0/wal/instance
I20260808 00:47:19.173058 20901 external_mini_cluster.cc:1428] Setting key 5b2c06b105a432a130ca46b5765676b2
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:47:19.182134 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.170763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43579 (local address 127.0.0.1:38418)
0808 00:47:19.170958 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.171004 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.171582 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.171879 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.171882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.172263 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.172489 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.172494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.174099 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.174100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.174849 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.174851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.175607 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.175609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.175712 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.176439 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.176452 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.178292 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.180405 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.180412 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.180415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.180629 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.180906 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.180907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.180907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.181005 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.181426 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.181432 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.181500 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.181627 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.181891 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":95,"thread_start_us":77,"threads_started":1}
I20260808 00:47:19.182139 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.170950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38418 (local address 127.20.105.126:43579)
0808 00:47:19.171100 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.171104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.171137 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:47:19.171737 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.171739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.171758 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.171848 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.172610 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.173964 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.174936 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.175482 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.175622 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.178405 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.178416 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.178416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.178429 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.180307 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.180733 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.180736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.180737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.180831 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.181091 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.181092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.181093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.181350 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.181454 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.181664 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.181822 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":73,"threads_started":1}
I20260808 00:47:19.183115 21902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:19.183594 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.183457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33550)
0808 00:47:19.183512 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.183566 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 00:47:19.183746 21810 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.105.125@KRBTEST.COM; defaulting to no policy
I20260808 00:47:19.184239 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.184138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33560)
0808 00:47:19.184184 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.184210 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:47:19.184396 21902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:35253: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab.
Entry for principal kudu/127.20.105.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab.
I20260808 00:47:19.207755 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.125:35253
--webserver_interface=127.20.105.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab
--principal=kudu/127.20.105.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:38067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:43579,127.20.105.125:35253,127.20.105.124:38263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:47:19.233697 21902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } attempt: 1
I20260808 00:47:19.234365 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.234182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33568)
0808 00:47:19.234267 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.234312 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 00:47:19.234776 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.234672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33580)
0808 00:47:19.234718 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.234752 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:47:19.234874 21902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:35253: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
I20260808 00:47:19.303078 21902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } attempt: 2
I20260808 00:47:19.303751 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.303568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33596)
0808 00:47:19.303659 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.303710 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 00:47:19.304230 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.304119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33612)
0808 00:47:19.304170 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.304207 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:47:19.304355 21902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.125:35253: Network error: Client connection negotiation failed: client connection to 127.20.105.125:35253: connect: Connection refused (error 111)
W20260808 00:47:19.311630 21910 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:47:19.311919 21910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:19.311964 21910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:19.312068 21910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:19.313335 21910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:19.313408 21910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:19.313444 21910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:19.313473 21910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:19.313498 21910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:19.315150 21910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:38067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:43579,127.20.105.125:35253,127.20.105.124:38263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.125.keytab
--principal=kudu/127.20.105.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.125:35253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.105.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:19.315480 21910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:19.315760 21910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:47:19.321568 21921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:19.321543 21919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:19.321581 21918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:19.321849 21910 server_base.cc:1034] running on GCE node
I20260808 00:47:19.322155 21910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:19.322491 21910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:19.323695 21910 hybrid_clock.cc:648] HybridClock initialized: now 1786150039323672 us; error 41 us; skew 500 ppm
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:19.326781 21910 init.cc:377] Logged in from keytab as kudu/127.20.105.125@KRBTEST.COM (short username kudu)
I20260808 00:47:19.327373 21910 webserver.cc:460] Webserver started at http://127.20.105.125:41483/ using document root <none> and password file <none>
I20260808 00:47:19.327695 21910 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:19.327775 21910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:19.327944 21910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:19.329353 21910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/data/instance:
uuid: "892408c5ae7b408fb196d242452d306d"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "6fc2435e0fcc2120264174b742b2c6e0"
server_key_iv: "1a617ebe40449f1315dee04d1babeef5"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.329798 21910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/wal/instance:
uuid: "892408c5ae7b408fb196d242452d306d"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "6fc2435e0fcc2120264174b742b2c6e0"
server_key_iv: "1a617ebe40449f1315dee04d1babeef5"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.331326 21910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:47:19.332243 21934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:19.332430 21910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:19.332561 21910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/wal
uuid: "892408c5ae7b408fb196d242452d306d"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "6fc2435e0fcc2120264174b742b2c6e0"
server_key_iv: "1a617ebe40449f1315dee04d1babeef5"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.332659 21910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:19.345862 21910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:19.348629 21910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:19.348816 21910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:19.353785 21910 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.125:35253
I20260808 00:47:19.353844 22021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.125:35253 every 8 connection(s)
I20260808 00:47:19.354287 21910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/data/info.pb
I20260808 00:47:19.355378 22022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:19.355767 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 21910
I20260808 00:47:19.355897 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-1/wal/instance
I20260808 00:47:19.356168 20901 external_mini_cluster.cc:1428] Setting key 45e8697425e60b0a0c6b5e9d6898ecca
I20260808 00:47:19.358250 22022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.105.124@KRBTEST.COM; defaulting to no policy
I20260808 00:47:19.370288 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.359298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38426 (local address 127.20.105.126:43579)
0808 00:47:19.359387 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.359391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.359686 (+ 295us) server_negotiation.cc:407] Connection header received
0808 00:47:19.360558 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.360559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.360578 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.360623 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.361588 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.362152 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.363832 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.364131 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.364197 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.366905 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.366908 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.366908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.366920 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.368573 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.369167 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.369169 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.369170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.369236 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.369517 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.369518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.369518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.369662 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.369741 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.370067 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.370171 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:47:19.370537 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.359263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43579 (local address 127.0.0.1:38426)
0808 00:47:19.359508 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.359568 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.360412 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.360720 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.360725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.361160 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.361491 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.361498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.362268 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.362270 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.363723 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.363727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.364228 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.364230 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.364419 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.365024 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.365037 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.366793 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.368685 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.368694 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.368697 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.369052 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.369311 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.369312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.369312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.369442 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.369744 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.369753 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.369919 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.370042 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.370238 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.105.124@KRBTEST.COM" created.
I20260808 00:47:19.371279 22022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:47:19.380405 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.371650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33624)
0808 00:47:19.371709 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.371722 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.371774 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.372153 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.372154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.372282 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.372345 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.372346 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.373633 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.373634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.374515 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.374518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.375120 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.375122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.375208 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.375674 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.375680 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.377246 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.379177 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.379178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.379179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.379356 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.379597 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.379598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.379598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.379662 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.379981 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.379982 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.380078 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.380151 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.380274 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:47:19.380424 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.371714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33624 (local address 127.20.105.125:35253)
0808 00:47:19.371934 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.371941 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.371969 (+ 28us) server_negotiation.cc:407] Connection header received
0808 00:47:19.371996 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.371999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.372014 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.372140 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.372425 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.373522 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.374623 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.375030 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.375107 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.377326 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.377338 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.377338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.377346 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.379066 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.379446 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.379450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.379450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.379512 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.379726 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.379727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.379727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.379905 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.380012 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.380184 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.380328 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":178,"thread_start_us":119,"threads_started":1}
I20260808 00:47:19.381207 22022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:19.381661 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.381505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44412)
0808 00:47:19.381573 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.381632 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 00:47:19.381839 21940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
I20260808 00:47:19.382318 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.382213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44420)
0808 00:47:19.382264 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.382289 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:47:19.382434 22022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:38263: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.105.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab.
Entry for principal kudu/127.20.105.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab.
I20260808 00:47:19.393128 20901 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskzO4qSW/build/release/bin/kudu
/tmp/dist-test-taskzO4qSW/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.124:38263
--webserver_interface=127.20.105.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab
--principal=kudu/127.20.105.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.105.84:38067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.105.126:43579,127.20.105.125:35253,127.20.105.124:38263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:47:19.395534 21902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } attempt: 3
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:47:19.405313 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.396078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33626)
0808 00:47:19.396164 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.396177 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.396255 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.396425 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.396426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.396635 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.396725 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.396727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.397743 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.397744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.398356 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.398359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.398992 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.398993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.399102 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.399760 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.399766 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.401652 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.404069 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.404070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.404071 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.404287 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.404539 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.404540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.404541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.404609 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.404997 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.404998 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.405055 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.405116 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.405220 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 00:47:19.405439 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.396186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33626 (local address 127.20.105.125:35253)
0808 00:47:19.396270 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.396272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.396287 (+ 15us) server_negotiation.cc:407] Connection header received
0808 00:47:19.396327 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.396328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.396341 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.396419 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.396820 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.397617 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.398462 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.398870 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.398972 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.401770 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.401772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.401773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.401784 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.403893 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.404376 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.404378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.404379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.404455 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.404694 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.404695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.404696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.404880 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.404963 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.405192 (+ 229us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.405294 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 00:47:19.405993 21902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:19.406445 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.406321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44424)
0808 00:47:19.406377 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.406417 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 00:47:19.406857 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.406783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44440)
0808 00:47:19.406815 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.406840 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 00:47:19.406975 21902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:38263: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
I20260808 00:47:19.431731 22022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } attempt: 1
I20260808 00:47:19.432585 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.432363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44448)
0808 00:47:19.432460 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.432523 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:47:19.433105 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.432988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44458)
0808 00:47:19.433046 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.433077 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 00:47:19.433238 22022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:38263: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
I20260808 00:47:19.438243 21902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } attempt: 1
I20260808 00:47:19.438755 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.438591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44460)
0808 00:47:19.438667 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.438718 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:47:19.439229 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.439105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44476)
0808 00:47:19.439170 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.439208 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 00:47:19.439323 21902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:38263: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
W20260808 00:47:19.496215 22034 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:47:19.496481 22034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:47:19.496500 22034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:47:19.496528 22034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:47:19.498070 22034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:47:19.498152 22034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:47:19.498169 22034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:47:19.498184 22034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:47:19.498198 22034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:47:19.500264 22034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.105.84:38067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.105.126:43579,127.20.105.125:35253,127.20.105.124:38263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/krb5kdc/kudu_127.20.105.124.keytab
--principal=kudu/127.20.105.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.105.124:38263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.105.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:47:19.500620 22034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:47:19.500981 22034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:47:19.501420 22022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } attempt: 2
I20260808 00:47:19.502216 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.502008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44484)
0808 00:47:19.502111 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.502169 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 00:47:19.502847 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.502691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44492)
0808 00:47:19.502763 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.502811 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 00:47:19.503001 22022 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:38263: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
W20260808 00:47:19.507917 22041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:19.507961 22039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:47:19.507972 22044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:47:19.508634 22034 server_base.cc:1034] running on GCE node
I20260808 00:47:19.508982 22034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:47:19.509382 22034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:47:19.510562 22034 hybrid_clock.cc:648] HybridClock initialized: now 1786150039510541 us; error 41 us; skew 500 ppm
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:47:19.513671 22034 init.cc:377] Logged in from keytab as kudu/127.20.105.124@KRBTEST.COM (short username kudu)
I20260808 00:47:19.514254 22034 webserver.cc:460] Webserver started at http://127.20.105.124:43789/ using document root <none> and password file <none>
I20260808 00:47:19.514521 22034 fs_manager.cc:359] Metadata directory not provided
I20260808 00:47:19.514463 21902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } attempt: 2
I20260808 00:47:19.514596 22034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:47:19.514770 22034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:47:19.515064 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.514854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44502)
0808 00:47:19.514970 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.515023 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 00:47:19.515571 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.515386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44518)
0808 00:47:19.515502 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.515536 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
W20260808 00:47:19.515708 21902 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.105.124:38263: Network error: Client connection negotiation failed: client connection to 127.20.105.124:38263: connect: Connection refused (error 111)
I20260808 00:47:19.516355 22034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/data/instance:
uuid: "92f5482b846f4d85aeb8748bdfef56cc"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "3caeb663f4235b14f802be41b7177d48"
server_key_iv: "1504de4da07fb7906e1f87731ac0467b"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.516767 22034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/wal/instance:
uuid: "92f5482b846f4d85aeb8748bdfef56cc"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "3caeb663f4235b14f802be41b7177d48"
server_key_iv: "1504de4da07fb7906e1f87731ac0467b"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.518349 22034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:47:19.519352 22054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:47:19.519603 22034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:47:19.519719 22034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/data,/tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/wal
uuid: "92f5482b846f4d85aeb8748bdfef56cc"
format_stamp: "Formatted at 2026-08-08 00:47:19 on dist-test-slave-kvfs"
server_key: "3caeb663f4235b14f802be41b7177d48"
server_key_iv: "1504de4da07fb7906e1f87731ac0467b"
server_key_version: "encryptionkey@0"
I20260808 00:47:19.519814 22034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:47:19.543686 22034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:47:19.546681 22034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:47:19.546911 22034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:47:19.552338 22034 rpc_server.cc:307] RPC server started. Bound to: 127.20.105.124:38263
I20260808 00:47:19.552368 22136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.105.124:38263 every 8 connection(s)
I20260808 00:47:19.552879 22034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/data/info.pb
I20260808 00:47:19.554221 22137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:47:19.556550 22137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:47:19.562420 20901 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskzO4qSW/build/release/bin/kudu as pid 22034
I20260808 00:47:19.562582 20901 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskzO4qSW/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149729697709-20901-0/minicluster-data/master-2/wal/instance
I20260808 00:47:19.562811 20901 external_mini_cluster.cc:1428] Setting key 16849c49de09713ed228946b9d3d5762
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
I20260808 00:47:19.567943 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.557595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38428 (local address 127.20.105.126:43579)
0808 00:47:19.557698 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.557701 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.558001 (+ 300us) server_negotiation.cc:407] Connection header received
0808 00:47:19.558570 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.558572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.558586 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.558656 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.559399 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.559972 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.561097 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.561320 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.561391 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.564481 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.564483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.564484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.564495 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.566327 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.566832 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.566833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.566834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.566913 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.567178 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.567179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.567179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.567317 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.567407 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.567720 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.567820 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 00:47:19.568188 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.557548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.126:43579 (local address 127.0.0.1:38428)
0808 00:47:19.557832 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.557880 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.558452 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.558749 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.558752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.559073 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.559316 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.559320 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.560129 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.560130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.561001 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.561004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.561444 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.561446 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.561578 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.562320 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.562331 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.564356 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.566475 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.566483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.566486 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.566746 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.566987 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.566988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.566988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.567107 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.567398 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.567406 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.567558 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.567691 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.567864 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":142,"threads_started":1}
I20260808 00:47:19.568941 22137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:47:19.573601 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.563397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44522 (local address 127.20.105.124:38263)
0808 00:47:19.563616 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.563622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.563651 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:47:19.563716 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.563718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.563744 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.563857 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.564787 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.565604 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.566521 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.566908 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.567016 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.570084 (+ 3068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.570093 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.570094 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.570103 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.572208 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.572618 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.572622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.572623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.572692 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.572922 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.572922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.572923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.573124 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.573328 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.573382 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.573476 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:47:19.576942 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.569272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.125:35253 (local address 127.0.0.1:33628)
0808 00:47:19.569341 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.569354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.569399 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.569633 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.569635 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.569793 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.569847 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.569848 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.570908 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.570909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.571425 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.571426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.571983 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.571984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.572123 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.572592 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.572598 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.573999 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.575820 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.575821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.575821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.576000 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.576256 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.576257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.576258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.576322 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.576597 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.576598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.576641 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.576731 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.576826 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 00:47:19.576942 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.569413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33628 (local address 127.20.105.125:35253)
0808 00:47:19.569489 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.569491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.569500 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:47:19.569535 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.569536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.569552 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.569630 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.569942 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.570801 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.571513 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.571878 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.571976 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.574093 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.574095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.574095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.574105 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.575692 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.576120 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.576122 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.576122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.576186 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.576405 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.576406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.576406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.576527 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.576607 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.576758 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.576839 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:47:19.577541 22137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.126@KRBTEST.COM
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.124@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:47:19.582940 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.574672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.20.105.126:43579)
0808 00:47:19.574728 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.574730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.574739 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:47:19.574810 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.574811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.574823 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.574893 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.575454 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.575987 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.576813 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.577101 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.577183 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.579696 (+ 2513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.579698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.579698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.579706 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.581528 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.581918 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.581919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.581919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.581983 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.582225 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.582226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.582226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.582404 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.582670 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.582721 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.582821 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:47:19.584868 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.577832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44528)
0808 00:47:19.577887 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.577897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.577934 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.578042 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.578043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.578179 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.578205 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.578206 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.578878 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.578879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.579481 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.579483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.579935 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.579936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.579998 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.580350 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.580356 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.581787 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.583717 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.583718 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.583718 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.583923 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.584197 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.584198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.584199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.584269 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.584507 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.584508 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.584563 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.584651 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.584752 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:47:19.584959 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.577906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44528 (local address 127.20.105.124:38263)
0808 00:47:19.577944 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.577946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.577956 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:47:19.577984 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.577985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.577994 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.578042 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.578286 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.578792 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.579565 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.579844 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.579904 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.581901 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.581902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.581903 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.581912 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.583633 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.584005 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.584006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.584007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.584106 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.584329 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.584330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.584330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.584440 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.584493 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.584669 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.584839 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":23}
I20260808 00:47:19.587191 22137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc: Bootstrap starting.
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.105.125@KRBTEST.COM
I20260808 00:47:19.588471 22137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:19.588841 22137 log.cc:824] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:19.590173 22137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc: No bootstrap required, opened a new log
I20260808 00:47:19.591841 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.583904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33644 (local address 127.20.105.125:35253)
0808 00:47:19.583977 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.583979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.584008 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:47:19.584114 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.584116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.584130 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.584191 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.584819 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.585435 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.586262 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.586491 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.586571 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.588941 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.588943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.588943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.588951 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.590624 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.590983 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.590984 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.590984 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.591051 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.591247 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.591248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.591249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.591412 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.591608 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.591649 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.591734 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:47:19.592566 22137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } }
I20260808 00:47:19.592756 22137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f5482b846f4d85aeb8748bdfef56cc, State: Initialized, Role: FOLLOWER
I20260808 00:47:19.592860 22137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } }
I20260808 00:47:19.593250 22137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:19.593202 22147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.593324 22147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:19.593791 20901 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:47:19.594177 22022 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } attempt: 3
W20260808 00:47:19.595382 22162 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:47:19.595443 22162 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:47:19.595423 21858 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "764cf92450844d4083faa765eee45470"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38436
I20260808 00:47:19.595611 22034 master_runner.cc:186] Error getting master registration for 127.20.105.126:43579: OK, instance_id { permanent_uuid: "764cf92450844d4083faa765eee45470" instance_seqno: 1786150039164707 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:47:19.598603 22034 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.125@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:47:19.603427 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.594598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.20.105.124:38263)
0808 00:47:19.594648 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.594650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.594663 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:47:19.594691 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.594691 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.594704 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.594771 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.595196 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.596065 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.597091 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.597474 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.597546 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.600153 (+ 2607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.600154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.600155 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.600161 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.601981 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.602427 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.602429 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.602430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.602501 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.602768 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.602769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.602769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.602957 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.603025 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.603229 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.603319 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:47:19.603446 22028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.594477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44542)
0808 00:47:19.594585 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.594599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.594639 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.594827 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.594828 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.595032 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.595092 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.595094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.596170 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.596171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.596996 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.596999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.597576 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.597577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.597645 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.598225 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.598231 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.599999 (+ 1768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.602084 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.602086 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.602086 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.602346 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.602596 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.602597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.602597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.602686 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.603035 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.603037 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.603096 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.603196 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.603323 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 00:47:19.606032 22022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d: Bootstrap starting.
I20260808 00:47:19.606945 22022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:19.607308 22022 log.cc:824] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:19.608438 22022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d: No bootstrap required, opened a new log
I20260808 00:47:19.610703 22022 raft_consensus.cc:348] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } }
I20260808 00:47:19.610872 22022 raft_consensus.cc:729] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 892408c5ae7b408fb196d242452d306d, State: Initialized, Role: FOLLOWER
I20260808 00:47:19.610994 22022 consensus_queue.cc:260] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } }
I20260808 00:47:19.611426 22022 sys_catalog.cc:564] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:19.611370 22165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.611626 22165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:47:19.613756 22182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:47:19.613876 22182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:47:19.616115 21902 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } attempt: 3
I20260808 00:47:19.614104 21910 master_runner.cc:186] Error getting master registration for 127.20.105.126:43579: OK, instance_id { permanent_uuid: "764cf92450844d4083faa765eee45470" instance_seqno: 1786150039164707 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:47:19.617563 21910 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 00:47:19 dist-test-slave-kvfs krb5kdc[21763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786150039, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.105.126@KRBTEST.COM for kudu/127.20.105.124@KRBTEST.COM
I20260808 00:47:19.627228 21904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.616512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.105.124:38263 (local address 127.0.0.1:44554)
0808 00:47:19.617691 (+ 1179us) negotiation.cc:107] Waiting for socket to connect
0808 00:47:19.617707 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 00:47:19.617790 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:47:19.617967 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:47:19.617969 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:47:19.618216 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:47:19.618289 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.618292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.619390 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.619391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.620312 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:47:19.620316 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.620856 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.620857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:47:19.620928 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.621538 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:47:19.621544 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:47:19.623465 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:47:19.625922 (+ 2457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.625923 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.625924 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.626179 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.626418 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:47:19.626418 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:47:19.626419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:47:19.626503 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:47:19.626837 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:47:19.626838 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:47:19.626903 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 00:47:19.626977 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:47:19.627099 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1146}
I20260808 00:47:19.627239 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.616673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.20.105.124:38263)
0808 00:47:19.616760 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.616763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.617768 (+ 1005us) server_negotiation.cc:407] Connection header received
0808 00:47:19.617872 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.617874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.617888 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.617961 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.618365 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.619271 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.620397 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.620755 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.620834 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.623593 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.623595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.623595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.623606 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.625802 (+ 2196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.626267 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.626268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.626269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.626344 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.626581 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.626582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.626582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.626752 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.626838 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.627024 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.627121 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 00:47:19.629961 21902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470: Bootstrap starting.
I20260808 00:47:19.630918 21902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470: Neither blocks nor log segments found. Creating new log.
I20260808 00:47:19.631278 21902 log.cc:824] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470: Log is configured to *not* fsync() on all Append() calls
I20260808 00:47:19.632396 21902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470: No bootstrap required, opened a new log
I20260808 00:47:19.634143 21902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } }
I20260808 00:47:19.634418 21902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 764cf92450844d4083faa765eee45470, State: Initialized, Role: FOLLOWER
I20260808 00:47:19.634631 21902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } }
I20260808 00:47:19.635444 21858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 00:47:19.635538 21858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:19.636835 21858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } }
I20260808 00:47:19.637322 21858 leader_election.cc:288] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [CANDIDATE]: Term 1 election: Requested vote from peers 892408c5ae7b408fb196d242452d306d (127.20.105.125:35253), 92f5482b846f4d85aeb8748bdfef56cc (127.20.105.124:38263)
I20260808 00:47:19.637457 21902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:47:19.637614 21987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "764cf92450844d4083faa765eee45470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "892408c5ae7b408fb196d242452d306d"
I20260808 00:47:19.637790 21987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:47:19.638929 21987 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 764cf92450844d4083faa765eee45470 in term 1.
I20260808 00:47:19.639235 21810 leader_election.cc:302] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 764cf92450844d4083faa765eee45470, 892408c5ae7b408fb196d242452d306d; no voters:
I20260808 00:47:19.639839 22101 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "764cf92450844d4083faa765eee45470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "92f5482b846f4d85aeb8748bdfef56cc"
I20260808 00:47:19.639994 22101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [term 0 FOLLOWER]: Advancing to term 1
W20260808 00:47:19.640198 22208 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:47:19.640257 22208 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:47:19.641069 22101 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 764cf92450844d4083faa765eee45470 in term 1.
I20260808 00:47:19.641659 22209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.641798 22209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:19.641916 22209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:47:19.642015 22209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [term 1 LEADER]: Becoming Leader. State: Replica: 764cf92450844d4083faa765eee45470, State: Running, Role: LEADER
I20260808 00:47:19.642160 22209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } }
I20260808 00:47:19.642560 21784 master_runner.cc:263] No leader master found from master 764cf92450844d4083faa765eee45470
I20260808 00:47:19.642627 21784 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:47:19.642797 22209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 764cf92450844d4083faa765eee45470. Latest consensus state: current_term: 1 leader_uuid: "764cf92450844d4083faa765eee45470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.642935 22209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:19.643673 22214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:47:19.643942 22214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:47:19.653518 21987 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [term 1 FOLLOWER]: Refusing update from remote peer 764cf92450844d4083faa765eee45470: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:47:19.653518 22101 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [term 1 FOLLOWER]: Refusing update from remote peer 764cf92450844d4083faa765eee45470: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:47:19.653981 22186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [LEADER]: Connected to new peer: Peer: permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:47:19.654183 22186 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:47:19.655822 22147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 764cf92450844d4083faa765eee45470. Latest consensus state: current_term: 1 leader_uuid: "764cf92450844d4083faa765eee45470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.655839 22165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 764cf92450844d4083faa765eee45470. Latest consensus state: current_term: 1 leader_uuid: "764cf92450844d4083faa765eee45470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.655951 22165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:19.655944 22147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:19.656960 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.646244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.20.105.124:38263)
0808 00:47:19.646303 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.646305 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.647404 (+ 1099us) server_negotiation.cc:407] Connection header received
0808 00:47:19.647508 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.647510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.647521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.647551 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.648253 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.649017 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.650299 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.650628 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.650675 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.652591 (+ 1916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.652593 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.652593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.652602 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.654873 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.655699 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.655701 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.655701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.655769 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.656297 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.656299 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.656299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.656444 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.656528 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.656744 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.656830 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:47:19.658643 22209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "764cf92450844d4083faa765eee45470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.658752 22209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:19.659035 22215 mvcc.cc:205] Tried to move back new op lower bound from 7316070562417008640 to 7316070562375598080. Current Snapshot: MvccSnapshot[applied={T|T < 7316070562417008640}]
I20260808 00:47:19.659317 22147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "764cf92450844d4083faa765eee45470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.659413 22147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:19.659667 22214 catalog_manager.cc:1347] Generated new cluster ID: a7d702bd8cb94ce5bf1196c6fcdcfae1
I20260808 00:47:19.659722 22214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:47:19.660458 22226 mvcc.cc:205] Tried to move back new op lower bound from 7316070562417008640 to 7316070562375598080. Current Snapshot: MvccSnapshot[applied={T|T < 7316070562417008640}]
I20260808 00:47:19.661818 22186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "764cf92450844d4083faa765eee45470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.661913 22186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470 [sys.catalog]: This master's current role is: LEADER
I20260808 00:47:19.662257 22165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "764cf92450844d4083faa765eee45470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "764cf92450844d4083faa765eee45470" member_type: VOTER last_known_addr { host: "127.20.105.126" port: 43579 } } peers { permanent_uuid: "892408c5ae7b408fb196d242452d306d" member_type: VOTER last_known_addr { host: "127.20.105.125" port: 35253 } } peers { permanent_uuid: "92f5482b846f4d85aeb8748bdfef56cc" member_type: VOTER last_known_addr { host: "127.20.105.124" port: 38263 } } }
I20260808 00:47:19.662357 22165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:47:19.664506 22222 mvcc.cc:205] Tried to move back new op lower bound from 7316070562417008640 to 7316070562375598080. Current Snapshot: MvccSnapshot[applied={T|T < 7316070562417008640}]
I20260808 00:47:19.666997 22029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.645787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.20.105.125:35253)
0808 00:47:19.645847 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.645849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.646395 (+ 546us) server_negotiation.cc:407] Connection header received
0808 00:47:19.646473 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.646474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.646490 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.646561 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.647568 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.648530 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.651909 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.652290 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.652341 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.660624 (+ 8283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.660626 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.660626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.660638 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.663019 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.663525 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.663526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.663527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.663599 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.666279 (+ 2680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.666280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.666280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.666451 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.666545 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.666790 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.666888 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:47:19.669098 21905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:19.646080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38450 (local address 127.20.105.126:43579)
0808 00:47:19.646128 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.646130 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.646555 (+ 425us) server_negotiation.cc:407] Connection header received
0808 00:47:19.646649 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.646650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.646664 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.646694 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.649906 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.650743 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.656143 (+ 5400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:19.656496 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:19.656546 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:19.663852 (+ 7306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:47:19.663854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:47:19.663854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:47:19.663863 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:47:19.665815 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.668008 (+ 2193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.668009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.668009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.668121 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:47:19.668389 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:47:19.668390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:47:19.668390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:47:19.668542 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:47:19.668649 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:19.668883 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 00:47:19.668982 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:47:19.684185 22214 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:47:19.685177 22214 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:47:19.695539 22214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 764cf92450844d4083faa765eee45470: Generated new TSK 0
I20260808 00:47:19.696084 22214 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:47:19.730757 21905 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.728977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.20.105.126:43579)
0808 00:47:19.729050 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.729052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.729839 (+ 787us) server_negotiation.cc:407] Connection header received
0808 00:47:19.729974 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.729976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.729991 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.730028 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.730651 (+ 623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38464: BlockingRecv error: recv got EOF from 127.0.0.1:38464 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260808 00:47:19.732314 22144 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.729492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44568 (local address 127.20.105.124:38263)
0808 00:47:19.729543 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.729546 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.731526 (+ 1980us) server_negotiation.cc:407] Connection header received
0808 00:47:19.731767 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.731769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.731783 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.731815 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.732239 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44568: BlockingRecv error: recv got EOF from 127.0.0.1:44568 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260808 00:47:19.733201 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.731349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33654 (local address 127.20.105.125:35253)
0808 00:47:19.731409 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.731411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.731641 (+ 230us) server_negotiation.cc:407] Connection header received
0808 00:47:19.732681 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.732682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.732696 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.732729 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.733128 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33654: BlockingRecv error: recv got EOF from 127.0.0.1:33654 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260808 00:47:19.735641 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.734801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33666 (local address 127.20.105.125:35253)
0808 00:47:19.734839 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.734841 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.735033 (+ 192us) server_negotiation.cc:407] Connection header received
0808 00:47:19.735159 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.735161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.735188 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.735220 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.735590 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33666: BlockingRecv error: recv got EOF from 127.0.0.1:33666 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 00:47:19.739248 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.738331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.20.105.125:35253)
0808 00:47:19.738368 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.738370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.738659 (+ 289us) server_negotiation.cc:407] Connection header received
0808 00:47:19.738763 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.738764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.738775 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.738806 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.739202 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33678: BlockingRecv error: recv got EOF from 127.0.0.1:33678 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:47:19.743451 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.742597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33694 (local address 127.20.105.125:35253)
0808 00:47:19.742634 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.742636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.742868 (+ 232us) server_negotiation.cc:407] Connection header received
0808 00:47:19.742970 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.742971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.742983 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.743014 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.743402 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33694: BlockingRecv error: recv got EOF from 127.0.0.1:33694 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:47:19.749841 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.748956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.20.105.125:35253)
0808 00:47:19.748996 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.748998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.749255 (+ 257us) server_negotiation.cc:407] Connection header received
0808 00:47:19.749365 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.749366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.749380 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.749410 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.749791 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33700: BlockingRecv error: recv got EOF from 127.0.0.1:33700 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260808 00:47:19.760556 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.759487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33716 (local address 127.20.105.125:35253)
0808 00:47:19.759538 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.759540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.759833 (+ 293us) server_negotiation.cc:407] Connection header received
0808 00:47:19.759953 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.759955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.759970 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.760004 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.760489 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33716: BlockingRecv error: recv got EOF from 127.0.0.1:33716 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:47:19.779570 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.778427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33722 (local address 127.20.105.125:35253)
0808 00:47:19.778485 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.778488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.778805 (+ 317us) server_negotiation.cc:407] Connection header received
0808 00:47:19.778925 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.778927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.778943 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.778979 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.779492 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33722: BlockingRecv error: recv got EOF from 127.0.0.1:33722 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 00:47:19.814527 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.813523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.20.105.125:35253)
0808 00:47:19.813588 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.813591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.813776 (+ 185us) server_negotiation.cc:407] Connection header received
0808 00:47:19.813876 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.813878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.813901 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.813973 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.814417 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33726: BlockingRecv error: recv got EOF from 127.0.0.1:33726 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260808 00:47:19.881160 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:19.880163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33732 (local address 127.20.105.125:35253)
0808 00:47:19.880253 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:19.880257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:19.880374 (+ 117us) server_negotiation.cc:407] Connection header received
0808 00:47:19.880454 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:19.880463 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:19.880477 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:19.880525 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:19.881061 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33732: BlockingRecv error: recv got EOF from 127.0.0.1:33732 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 00:47:20.011835 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:20.010780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33734 (local address 127.20.105.125:35253)
0808 00:47:20.010858 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:20.010860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:20.010967 (+ 107us) server_negotiation.cc:407] Connection header received
0808 00:47:20.011020 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:20.011022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:20.011038 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:20.011113 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:20.011712 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33734: BlockingRecv error: recv got EOF from 127.0.0.1:33734 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 00:47:20.270238 22029 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:47:20.269347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.20.105.125:35253)
0808 00:47:20.269435 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:20.269437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:20.269522 (+ 85us) server_negotiation.cc:407] Connection header received
0808 00:47:20.269612 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:20.269613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:20.269627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:20.269689 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:47:20.270128 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33748: BlockingRecv error: recv got EOF from 127.0.0.1:33748 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 00:47:20.596150 22162 catalog_manager.cc:1259] Loaded cluster ID: a7d702bd8cb94ce5bf1196c6fcdcfae1
I20260808 00:47:20.596266 22162 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc: loading cluster ID for follower catalog manager: success
I20260808 00:47:20.597206 22162 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc: acquiring CA information for follower catalog manager: success
I20260808 00:47:20.597546 22162 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 92f5482b846f4d85aeb8748bdfef56cc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:47:20.614555 22182 catalog_manager.cc:1259] Loaded cluster ID: a7d702bd8cb94ce5bf1196c6fcdcfae1
I20260808 00:47:20.614660 22182 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d: loading cluster ID for follower catalog manager: success
I20260808 00:47:20.615654 22182 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d: acquiring CA information for follower catalog manager: success
I20260808 00:47:20.616010 22182 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 892408c5ae7b408fb196d242452d306d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:47:20.787055 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:20.783991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.20.105.125:35253)
0808 00:47:20.784181 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:20.784183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:20.784255 (+ 72us) server_negotiation.cc:407] Connection header received
0808 00:47:20.784341 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:20.784342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:20.784429 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:20.784534 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:20.784716 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.785305 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.786140 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.786351 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.786414 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:20.786586 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:20.786742 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:20.786812 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:20.786846 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 00:47:20.786868 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":123,"threads_started":1}
I20260808 00:47:20.792116 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:20.789426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.20.105.124:38263)
0808 00:47:20.789599 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:20.789602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:20.789610 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:47:20.789665 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:20.789666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:20.789717 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:20.789805 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:20.789943 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.790469 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.791195 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.791397 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.791452 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:20.791657 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:20.791795 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:20.791860 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:20.791925 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 00:47:20.791945 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":84,"threads_started":1}
I20260808 00:47:20.797787 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:20.794481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38470 (local address 127.20.105.126:43579)
0808 00:47:20.794647 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:20.794649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:20.794725 (+ 76us) server_negotiation.cc:407] Connection header received
0808 00:47:20.794882 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:20.794883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:20.794966 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:20.795065 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:20.795296 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.795842 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.796832 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.797064 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.797152 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:20.797390 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:20.797486 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:20.797565 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:20.797639 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 00:47:20.797666 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260808 00:47:20.798082 22345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:20.794699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44582 (local address 127.20.105.124:38263)
0808 00:47:20.794738 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:20.794741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:20.794920 (+ 179us) server_negotiation.cc:407] Connection header received
0808 00:47:20.795030 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:20.795031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:20.795043 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:20.795098 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:20.795238 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.795944 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.797169 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.797463 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.797502 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:20.797713 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:20.797746 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:20.797777 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:20.797986 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 00:47:20.798009 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:47:20.798483 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:47:20.794386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33766 (local address 127.20.105.125:35253)
0808 00:47:20.794445 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:47:20.794447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:47:20.794793 (+ 346us) server_negotiation.cc:407] Connection header received
0808 00:47:20.794872 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:47:20.794874 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:47:20.794888 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:47:20.794936 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:47:20.795124 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.795898 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.797170 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:47:20.797464 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:47:20.797502 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:47:20.797713 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:47:20.797746 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:47:20.797777 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 00:47:20.798368 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 00:47:20.798404 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:47:20.800109 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21784
I20260808 00:47:20.825083 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 21910
I20260808 00:47:20.845050 20901 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskzO4qSW/build/release/bin/kudu with pid 22034
2026-08-08T00:47:20Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3264 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6735 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (311145 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST